Wenhui Wang

According to our database, Wenhui Wang authored at least 32 papers between 2017 and 2024.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of four.

2024

Fine-tuning pretrained transformer encoders for sequence-to-sequence learning.

Hangbo Bao, Li Dong, Wenhui Wang, Nan Yang, Songhao Piao, Furu Wei

Int. J. Mach. Learn. Cybern., May, 2024

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits.

Shuming Ma, Hongyu Wang, Lingxiao Ma, Lei Wang, Wenhui Wang, Shaohan Huang, Li Dong, Ruiping Wang, Jilong Xue, Furu Wei

CoRR, 2024

2023

When an Image is Worth 1, 024 x 1, 024 Words: A Case Study in Computational Pathology.

Wenhui Wang, Shuming Ma, Hanwen Xu, Naoto Usuyama, Jiayu Ding, Hoifung Poon, Furu Wei

CoRR, 2023

Kosmos-2.5: A Multimodal Literate Model.

Tengchao Lv, Yupan Huang, Jingye Chen, Lei Cui, Shuming Ma, Yaoyao Chang, Shaohan Huang, Wenhui Wang, Li Dong, Weiyao Luo, Shaoxiang Wu, Guoxin Wang, Cha Zhang, Furu Wei

CoRR, 2023

LongNet: Scaling Transformers to 1, 000, 000, 000 Tokens.

Jiayu Ding, Shuming Ma, Li Dong, Xingxing Zhang, Shaohan Huang, Wenhui Wang, Nanning Zheng, Furu Wei

CoRR, 2023

Kosmos-2: Grounding Multimodal Large Language Models to the World.

Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, Furu Wei

CoRR, 2023

Language Is Not All You Need: Aligning Perception with Language Models.

Shaohan Huang, Li Dong, Wenhui Wang, Yaru Hao, Saksham Singhal, Shuming Ma, Tengchao Lv, Lei Cui, Owais Khan Mohammed, Barun Patra, Qiang Liu, Kriti Aggarwal, Zewen Chi, Nils Johan Bertil Bjorck, Vishrav Chaudhary, Subhojit Som, Xia Song, Furu Wei

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

Magneto: A Foundation Transformer.

Hongyu Wang, Shuming Ma, Shaohan Huang, Li Dong, Wenhui Wang, Zhiliang Peng, Yu Wu, Payal Bajaj, Saksham Singhal, Alon Benhaim, Barun Patra, Zhun Liu, Vishrav Chaudhary, Xia Song, Furu Wei

Proceedings of the International Conference on Machine Learning, 2023

Image as a Foreign Language: BEIT Pretraining for Vision and Vision-Language Tasks.

Wenhui Wang, Hangbo Bao, Li Dong, Johan Bjorck, Zhiliang Peng, Qiang Liu, Kriti Aggarwal, Owais Khan Mohammed, Saksham Singhal, Subhojit Som, Furu Wei

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023

2022

TorchScale: Transformers at Scale.

Shuming Ma, Hongyu Wang, Shaohan Huang, Wenhui Wang, Zewen Chi, Li Dong, Alon Benhaim, Barun Patra, Vishrav Chaudhary, Xia Song, Furu Wei

CoRR, 2022

Foundation Transformers.

Hongyu Wang, Shuming Ma, Shaohan Huang, Li Dong, Wenhui Wang, Zhiliang Peng, Yu Wu, Payal Bajaj, Saksham Singhal, Alon Benhaim, Barun Patra, Zhun Liu, Vishrav Chaudhary, Xia Song, Furu Wei

CoRR, 2022

Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks.

Wenhui Wang, Hangbo Bao, Li Dong, Johan Bjorck, Zhiliang Peng, Qiang Liu, Kriti Aggarwal, Owais Khan Mohammed, Saksham Singhal, Subhojit Som, Furu Wei

CoRR, 2022

Language Models are General-Purpose Interfaces.

Yaru Hao, Haoyu Song, Li Dong, Shaohan Huang, Zewen Chi, Wenhui Wang, Shuming Ma, Furu Wei

CoRR, 2022

VL-BEiT: Generative Vision-Language Pretraining.

Hangbo Bao, Wenhui Wang, Li Dong, Furu Wei

CoRR, 2022

AutoDistil: Few-shot Task-agnostic Neural Architecture Search for Distilling Large Language Models.

Dongkuan Xu, Subhabrata Mukherjee, Xiaodong Liu, Debadeepta Dey, Wenhui Wang, Xiang Zhang, Ahmed Hassan Awadallah, Jianfeng Gao

CoRR, 2022

VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Experts.

Hangbo Bao, Wenhui Wang, Li Dong, Qiang Liu, Owais Khan Mohammed, Kriti Aggarwal, Subhojit Som, Songhao Piao, Furu Wei

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

Distilled Dual-Encoder Model for Vision-Language Understanding.

Zekun Wang, Wenhui Wang, Haichao Zhu, Ming Liu, Bing Qin, Furu Wei

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022

2021

VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Experts.

Wenhui Wang, Hangbo Bao, Li Dong, Furu Wei

CoRR, 2021

s2s-ft: Fine-Tuning Pretrained Transformer Encoders for Sequence-to-Sequence Learning.

Hangbo Bao, Li Dong, Wenhui Wang, Nan Yang, Furu Wei

CoRR, 2021

InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training.

Zewen Chi, Li Dong, Furu Wei, Nan Yang, Saksham Singhal, Wenhui Wang, Xia Song, Xian-Ling Mao, Heyan Huang, Ming Zhou

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2021

Consistency Regularization for Cross-Lingual Fine-Tuning.

Bo Zheng, Li Dong, Shaohan Huang, Wenhui Wang, Zewen Chi, Saksham Singhal, Wanxiang Che, Ting Liu, Xia Song, Furu Wei

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, 2021

Adapt-and-Distill: Developing Small, Fast and Effective Pretrained Language Models for Domains.

Yunzhi Yao, Shaohan Huang, Wenhui Wang, Li Dong, Furu Wei

Proceedings of the Findings of the Association for Computational Linguistics: ACL/IJCNLP 2021, 2021

MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers.

Wenhui Wang, Hangbo Bao, Shaohan Huang, Li Dong, Furu Wei

Proceedings of the Findings of the Association for Computational Linguistics: ACL/IJCNLP 2021, 2021

2020

MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers.

Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, Ming Zhou

Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, 2020

UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training.

Hangbo Bao, Li Dong, Furu Wei, Wenhui Wang, Nan Yang, Xiaodong Liu, Yu Wang, Jianfeng Gao, Songhao Piao, Ming Zhou, Hsiao-Wuen Hon

Proceedings of the 37th International Conference on Machine Learning, 2020

Harvesting and Refining Question-Answer Pairs for Unsupervised QA.

Zhongli Li, Wenhui Wang, Li Dong, Furu Wei, Ke Xu

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020

Cross-Lingual Natural Language Generation via Pre-Training.

Zewen Chi, Li Dong, Furu Wei, Wenhui Wang, Xian-Ling Mao, Heyan Huang

Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence, 2020

2019

Unified Language Model Pre-training for Natural Language Understanding and Generation.

Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, Hsiao-Wuen Hon

Proceedings of the Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, 2019

Learning to Ask Unanswerable Questions for Machine Reading Comprehension.

Haichao Zhu, Li Dong, Furu Wei, Wenhui Wang, Bing Qin, Ting Liu

Proceedings of the 57th Conference of the Association for Computational Linguistics, 2019

Inspecting Unification of Encoding and Matching with Transformer: A Case Study of Machine Reading Comprehension.

Hangbo Bao, Li Dong, Furu Wei, Wenhui Wang, Nan Yang, Lei Cui, Songhao Piao, Ming Zhou

Proceedings of the 2nd Workshop on Machine Reading for Question Answering, 2019

2018

Multiway Attention Networks for Modeling Sentence Pairs.

Chuanqi Tan, Furu Wei, Wenhui Wang, Weifeng Lv, Ming Zhou

Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, 2018

2017

Gated Self-Matching Networks for Reading Comprehension and Question Answering.

Wenhui Wang, Nan Yang, Furu Wei, Baobao Chang, Ming Zhou

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 2017