The Application of Leader-Member Exchange Theory on Improving Team Performance. - Shuming Ma; Yang Ke (ICEE, 2010)

A New Recurrent Neural CRF for Learning Non-linear Edge Features. - Shuming Ma; Xu Sun 001 (CoRR, 2016)

Improving Semantic Relevance for Sequence-to-Sequence Learning of Chinese Social Media Text Summarization. - Shuming Ma; Xu Sun 001; Jingjing Xu 001; Houfeng Wang; Wenjie Li 002; Qi Su 001 (ACL (2), 2017)

meProp: Sparsified Back Propagation for Accelerated Deep Learning with Reduced Overfitting. - Xu Sun 001; Xuancheng Ren; Shuming Ma; Houfeng Wang (ICML, 2017)

Transfer Deep Learning for Low-Resource Chinese Word Segmentation with a Novel Neural Network. - Jingjing Xu 001; Shuming Ma; Yi Zhang 050; Bingzhen Wei; Xiaoyan Cai; Xu Sun 001 (NLPCC, 2017)

A Generic Online Parallel Learning Framework for Large Margin Models. - Shuming Ma; Xu Sun 001 (CoRR, 2017)

Lock-Free Parallel Perceptron for Graph-based Dependency Parsing. - Xu Sun 001; Shuming Ma (CoRR, 2017)

A Semantic Relevance Based Neural Network for Text Summarization and Text Simplification. - Shuming Ma; Xu Sun 001 (CoRR, 2017)

Label Embedding Network: Learning Label Representation for Soft Training of Deep Networks. - Xu Sun 001; Bingzhen Wei; Xuancheng Ren; Shuming Ma (CoRR, 2017)

Stochastic Strictly Contractive Peaceman-Rachford Splitting Method. - Sen Na; Mingyuan Ma; Shuming Ma; Guangju Peng (CoRR, 2017)

Training Simplification and Model Simplification for Deep Learning: A Minimal Effort Back Propagation Method. - Xu Sun 001; Xuancheng Ren; Shuming Ma; Bingzhen Wei; Wei Li 101; Houfeng Wang (CoRR, 2017)

Complex Structure Leads to Overfitting: A Structure Regularization Decoding Method for Natural Language Processing. - Xu Sun 001; Weiwei Sun 001; Shuming Ma; Xuancheng Ren; Yi Zhang 050; Wenjie Li 002; Houfeng Wang (CoRR, 2017)

Global Encoding for Abstractive Summarization. - Junyang Lin; Xu Sun 001; Shuming Ma; Qi Su 001 (ACL (2), 2018)

Autoencoder as Assistant Supervisor: Improving Text Representation for Chinese Social Media Text Summarization. - Shuming Ma; Xu Sun 001; Junyang Lin; Houfeng Wang (ACL (2), 2018)

Bag-of-Words as Target for Neural Machine Translation. - Shuming Ma; Xu Sun 001; Yizhong Wang; Junyang Lin (ACL (2), 2018)

Automatic Academic Paper Rating Based on Modularized Hierarchical Convolutional Neural Network. - Pengcheng Yang; Xu Sun 001; Wei Li 101; Shuming Ma (ACL (2), 2018)

Deconvolution-Based Global Decoding for Neural Machine Translation. - Junyang Lin; Xu Sun 001; Xuancheng Ren; Shuming Ma; Jinsong Su; Qi Su 001 (COLING, 2018)

A Neural Question Answering Model Based on Semi-Structured Tables. - Hao Wang; Xiaodong Zhang 022; Shuming Ma; Xu Sun 001; Houfeng Wang; Mengxiang Wang (COLING, 2018)

SGM: Sequence Generation Model for Multi-label Classification. - Pengcheng Yang; Xu Sun 001; Wei Li 101; Shuming Ma; Wei Wu 044; Houfeng Wang (COLING, 2018)

Does Higher Order LSTM Have Better Accuracy for Segmenting and Labeling Sequence Data? - Yi Zhang 050; Xu Sun 001; Shuming Ma; Yang Yang 125; Xuancheng Ren (COLING, 2018)

Semantic-Unit-Based Dilated Convolution for Multi-Label Text Classification. - Junyang Lin; Qi Su 001; Pengcheng Yang; Shuming Ma; Xu Sun 001 (EMNLP, 2018)

Phrase-level Self-Attention Networks for Universal Sentence Encoding. - Wei Wu 044; Houfeng Wang; Tianyu Liu 001; Shuming Ma (EMNLP, 2018)

A Hierarchical End-to-End Model for Jointly Improving Text Summarization and Sentiment Classification. - Shuming Ma; Xu Sun 001; Junyang Lin; Xuancheng Ren (IJCAI, 2018)

Query and Output: Generating Words by Querying Distributed Word Representations for Paraphrase Generation. - Shuming Ma; Xu Sun 001; Wei Li 101; Sujian Li; Wenjie Li 002; Xuancheng Ren (NAACL-HLT, 2018)

Accelerating Graph-Based Dependency Parsing with Lock-Free Parallel Perceptron. - Shuming Ma; Xu Sun 001; Yi Zhang 050; Bingzhen Wei (NLPCC (1), 2018)

Decoding-History-Based Adaptive Control of Attention for Neural Machine Translation. - Junyang Lin; Shuming Ma; Qi Su 001; Xu Sun 001 (CoRR, 2018)

Identifying High-Quality Chinese News Comments Based on Multi-Target Text Matching Model. - Deli Chen; Shuming Ma; Pengcheng Yang; Xu Sun 001 (CoRR, 2018)

A Deep Reinforced Sequence-to-Set Model for Multi-Label Text Classification. - Pengcheng Yang; Shuming Ma; Yi Zhang 050; Junyang Lin; Qi Su 001; Xu Sun 001 (CoRR, 2018)

Unsupervised Machine Commenting with Neural Variational Topic Model. - Shuming Ma; Lei Cui 001; Furu Wei; Xu Sun 001 (CoRR, 2018)

Hierarchical Encoder with Auxiliary Supervision for Neural Table-to-Text Generation: Learning Better Representation for Tables. - Tianyu Liu 001; Fuli Luo; Qiaolin Xia; Shuming Ma; Baobao Chang; Zhifang Sui (AAAI, 2019)

LiveBot: Generating Live Video Comments Based on Visual and Textual Contexts. - Shuming Ma; Lei Cui 001; Damai Dai; Furu Wei; Xu Sun 001 (AAAI, 2019)

Key Fact as Pivot: A Two-Stage Model for Low Resource Table-to-Text Generation. - Shuming Ma; Pengcheng Yang; Tianyu Liu 001; Peng Li 030; Jie Zhou 016; Xu Sun 001 (ACL (1), 2019)

A Deep Reinforced Sequence-to-Set Model for Multi-Label Classification. - Pengcheng Yang; Fuli Luo; Shuming Ma; Junyang Lin; Xu Sun 001 (ACL (1), 2019)

Predicting Popular News Comments Based on Multi-Target Text Matching Model. - Deli Chen; Shuming Ma; Pengcheng Yang; Qi Su 001 (NLPCC (1), 2019)

Recursive Graphical Neural Networks for Text Classification. - Wei Li 101; Shuheng Li; Shuming Ma; Yancheng He; Deli Chen; Xu Sun 001 (CoRR, 2019)

Group, Extract and Aggregate: Summarizing a Large Amount of Finance News for Forex Movement Prediction. - Deli Chen; Shuming Ma; Keiko Harimoto; Ruihan Bao; Qi Su 001; Xu Sun 001 (CoRR, 2019)

Towards easier and faster sequence labeling for natural language processing: A search-based probabilistic online learning framework (SAPO). - Xu Sun 001; Shuming Ma; Yi Zhang 050; Xuancheng Ren (Inf. Sci., 2019)

Alternating Language Modeling for Cross-Lingual Pre-Training. - Jian Yang 030; Shuming Ma; Dongdong Zhang 001; Shuangzhi Wu; Zhoujun Li 001; Ming Zhou 001 (AAAI, 2020)

A Simple and Effective Unified Encoder for Document-Level Machine Translation. - Shuming Ma; Dongdong Zhang 001; Ming Zhou 001 (ACL, 2020)

Improving Neural Machine Translation with Soft Template Prediction. - Jian Yang 030; Shuming Ma; Dongdong Zhang 001; Zhoujun Li 001; Ming Zhou 001 (ACL, 2020)

Multimodal Matching Transformer for Live Commenting. - Chaoqun Duan; Lei Cui 001; Shuming Ma; Furu Wei; Conghui Zhu; Tiejun Zhao (ECAI, 2020)

XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders. - Shuming Ma; Jian Yang 030; Haoyang Huang; Zewen Chi; Li Dong 004; Dongdong Zhang 001; Hany Hassan Awadalla; Alexandre Muzio; Akiko Eriguchi; Saksham Singhal; Xia Song; Arul Menezes; et al. (CoRR, 2020)

Training Simplification and Model Simplification for Deep Learning : A Minimal Effort Back Propagation Method. - Xu Sun 001; Xuancheng Ren; Shuming Ma; Bingzhen Wei; Wei Li 101; Jingjing Xu 001; Houfeng Wang; Yi Zhang 050 (IEEE Trans. Knowl. Data Eng., 2020)

How Does Distilled Data Complexity Impact the Quality and Confidence of Non-Autoregressive Machine Translation? - Weijia Xu; Shuming Ma; Dongdong Zhang 001; Marine Carpuat (ACL/IJCNLP (Findings), 2021)

Multilingual Agreement for Multilingual Neural Machine Translation. - Jian Yang 030; Yuwei Yin; Shuming Ma; Haoyang Huang; Dongdong Zhang 001; Zhoujun Li 001; Furu Wei (ACL/IJCNLP (2), 2021)

Zero-Shot Cross-Lingual Transfer of Neural Machine Translation with Multilingual Pretrained Encoders. - Guanhua Chen 001; Shuming Ma; Yun Chen 007; Li Dong 004; Dongdong Zhang 001; Jia Pan 001; Wenping Wang 001; Furu Wei (EMNLP (1), 2021)

mT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs. - Zewen Chi; Li Dong 004; Shuming Ma; Shaohan Huang; Saksham Singhal; Xian-Ling Mao; Heyan Huang; Xia Song; Furu Wei (EMNLP (1), 2021)

Improving Multilingual Neural Machine Translation with Auxiliary Source Languages. - Weijia Xu; Yuwei Yin; Shuming Ma; Dongdong Zhang 001; Haoyang Huang (EMNLP (Findings), 2021)

Smart-Start Decoding for Neural Machine Translation. - Jian Yang 030; Shuming Ma; Dongdong Zhang 001; Juncheng Wan; Zhoujun Li 001; Ming Zhou 001 (NAACL-HLT, 2021)

Learning to Select Relevant Knowledge for Neural Machine Translation. - Jian Yang 030; Juncheng Wan; Shuming Ma; Haoyang Huang; Dongdong Zhang 001; Yong Yu 001; Zhoujun Li 001; Furu Wei (NLPCC (1), 2021)

Multilingual Machine Translation Systems from Microsoft for WMT21 Shared Task. - Jian Yang 030; Shuming Ma; Haoyang Huang; Dongdong Zhang 001; Li Dong 004; Shaohan Huang; Alexandre Muzio; Saksham Singhal; Hany Hassan; Xia Song; Furu Wei (WMT@EMNLP, 2021)

BlonD: An Automatic Evaluation Metric for Document-level MachineTranslation. - Yuchen Jiang; Shuming Ma; Dongdong Zhang 001; Jian Yang 030; Haoyang Huang; Ming Zhou 001 (CoRR, 2021)

mT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs. - Zewen Chi; Li Dong 004; Shuming Ma; Shaohan Huang; Xian-Ling Mao; Heyan Huang; Furu Wei (CoRR, 2021)

DeltaLM: Encoder-Decoder Pre-training for Language Generation and Translation by Augmenting Pretrained Multilingual Encoders. - Shuming Ma; Li Dong 004; Shaohan Huang; Dongdong Zhang 001; Alexandre Muzio; Saksham Singhal; Hany Hassan Awadalla; Xia Song; Furu Wei (CoRR, 2021)

XLM-E: Cross-lingual Language Model Pre-training via ELECTRA. - Zewen Chi; Shaohan Huang; Li Dong 004; Shuming Ma; Saksham Singhal; Payal Bajaj; Xia Song; Furu Wei (CoRR, 2021)

Towards Making the Most of Multilingual Pretraining for Zero-Shot Neural Machine Translation. - Guanhua Chen 001; Shuming Ma; Yun Chen 007; Dongdong Zhang 001; Jia Pan 001; Wenping Wang 001; Furu Wei (CoRR, 2021)

Towards Making the Most of Cross-Lingual Transfer for Zero-Shot Neural Machine Translation. - Guanhua Chen 001; Shuming Ma; Yun Chen 007; Dongdong Zhang 001; Jia Pan 001; Wenping Wang 001; Furu Wei (ACL (1), 2022)

XLM-E: Cross-lingual Language Model Pre-training via ELECTRA. - Zewen Chi; Shaohan Huang; Li Dong 004; Shuming Ma; Bo Zheng 010; Saksham Singhal; Payal Bajaj; Xia Song; Xian-Ling Mao; Heyan Huang; Furu Wei (ACL (1), 2022)

StableMoE: Stable Routing Strategy for Mixture of Experts. - Damai Dai; Li Dong 004; Shuming Ma; Bo Zheng 010; Zhifang Sui; Baobao Chang; Furu Wei (ACL (1), 2022)

PAEG: Phrase-level Adversarial Example Generation for Neural Machine Translation. - Juncheng Wan; Jian Yang 030; Shuming Ma; Dongdong Zhang 001; Weinan Zhang 001; Yong Yu 001; Zhoujun Li 001 (COLING, 2022)

Zero-shot Cross-lingual Transfer of Prompt-based Tuning with a Unified Multilingual Prompt. - Lianzhe Huang; Shuming Ma; Dongdong Zhang 001; Furu Wei; Houfeng Wang (EMNLP, 2022)

CROP: Zero-shot Cross-lingual Named Entity Recognition with Multilingual Labeled Sequence Translation. - Jian Yang 030; Shaohan Huang; Shuming Ma; Yuwei Yin; Li Dong 004; Dongdong Zhang 001; Hongcheng Guo; Zhoujun Li 001; Furu Wei (EMNLP (Findings), 2022)

A Unified Strategy for Multilingual Grammatical Error Correction with Pre-trained Cross-Lingual Language Model. - Xin Sun 013; Tao Ge 001; Shuming Ma; Jingjing Li 007; Furu Wei; Houfeng Wang (IJCAI, 2022)

High-resource Language-specific Training for Multilingual Neural Machine Translation. - Jian Yang 030; Yuwei Yin; Shuming Ma; Dongdong Zhang 001; Zhoujun Li 001; Furu Wei (IJCAI, 2022)

UM4: Unified Multilingual Multiple Teacher-Student Model for Zero-Resource Neural Machine Translation. - Jian Yang 030; Yuwei Yin; Shuming Ma; Dongdong Zhang 001; Shuangzhi Wu; Hongcheng Guo; Zhoujun Li 001; Furu Wei (IJCAI, 2022)

BlonDe: An Automatic Evaluation Metric for Document-level Machine Translation. - Yuchen Jiang; Tianyu Liu 004; Shuming Ma; Dongdong Zhang 001; Jian Yang 030; Haoyang Huang; Rico Sennrich; Ryan Cotterell; Mrinmaya Sachan; Ming Zhou 001 (NAACL-HLT, 2022)

On the Representation Collapse of Sparse Mixture of Experts. - Zewen Chi; Li Dong 004; Shaohan Huang; Damai Dai; Shuming Ma; Barun Patra; Saksham Singhal; Payal Bajaj; Xia Song; Xian-Ling Mao; Heyan Huang; Furu Wei (NeurIPS, 2022)

SMDT: Selective Memory-Augmented Neural Document Translation. - Xu Zhang; Jian Yang 030; Haoyang Huang; Shuming Ma; Dongdong Zhang 001; Jinlong Li; Furu Wei (CoRR, 2022)

Phrase-level Adversarial Example Generation for Neural Machine Translation. - Juncheng Wan; Jian Yang 030; Shuming Ma; Dongdong Zhang 001; Weinan Zhang 001; Yong Yu 001; Furu Wei (CoRR, 2022)

DeepNet: Scaling Transformers to 1, 000 Layers. - Hongyu Wang 009; Shuming Ma; Li Dong 004; Shaohan Huang; Dongdong Zhang 001; Furu Wei (CoRR, 2022)

On the Representation Collapse of Sparse Mixture of Experts. - Zewen Chi; Li Dong 004; Shaohan Huang; Damai Dai; Shuming Ma; Barun Patra; Saksham Singhal; Payal Bajaj; Xia Song; Furu Wei (CoRR, 2022)

Language Models are General-Purpose Interfaces. - Yaru Hao; Haoyu Song 002; Li Dong 004; Shaohan Huang; Zewen Chi; Wenhui Wang 003; Shuming Ma; Furu Wei (CoRR, 2022)

HLT-MT: High-resource Language-specific Training for Multilingual Neural Machine Translation. - Jian Yang 030; Yuwei Yin; Shuming Ma; Dongdong Zhang 001; Zhoujun Li 001; Furu Wei (CoRR, 2022)

GTrans: Grouping and Fusing Transformer Layers for Neural Machine Translation. - Jian Yang 030; Yuwei Yin; Shuming Ma; Haoyang Huang; Dongdong Zhang 001; Furu Wei; Zhoujun Li 001 (CoRR, 2022)

Towards Multilingual Transitivity and Bidirectional Multilingual Agreement for Multilingual Document-level Machine Translation. - Hongyuan Lu; Haoyang Huang; Shuming Ma; Dongdong Zhang 001; Furu Wei; Wai Lam (CoRR, 2022)

Foundation Transformers. - Hongyu Wang 009; Shuming Ma; Shaohan Huang; Li Dong 004; Wenhui Wang 003; Zhiliang Peng; Yu Wu; Payal Bajaj; Saksham Singhal; Alon Benhaim; Barun Patra; Zhun Liu; et al. (CoRR, 2022)

A Bilingual Parallel Corpus with Discourse Annotations. - Yuchen Eleanor Jiang; Tianyu Liu 004; Shuming Ma; Dongdong Zhang 001; Mrinmaya Sachan; Ryan Cotterell (CoRR, 2022)

TorchScale: Transformers at Scale. - Shuming Ma; Hongyu Wang 009; Shaohan Huang; Wenhui Wang 003; Zewen Chi; Li Dong 004; Alon Benhaim; Barun Patra; Vishrav Chaudhary; Xia Song; Furu Wei (CoRR, 2022)

TRIP: Triangular Document-level Pre-training for Multilingual Language Models. - Hongyuan Lu; Haoyang Huang; Shuming Ma; Dongdong Zhang 001; Wai Lam; Furu Wei (CoRR, 2022)

On the Off-Target Problem of Zero-Shot Multilingual Neural Machine Translation. - Liang Chen 024; Shuming Ma; Dongdong Zhang 001; Furu Wei; Baobao Chang (ACL (Findings), 2023)

Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers. - Damai Dai; Yutao Sun; Li Dong 004; Yaru Hao; Shuming Ma; Zhifang Sui; Furu Wei (ACL (Findings), 2023)

Discourse-Centric Evaluation of Document-level Machine Translation with a New Densely Annotated Parallel Corpus of Novels. - Yuchen Eleanor Jiang; Tianyu Liu 004; Shuming Ma; Dongdong Zhang 001; Mrinmaya Sachan; Ryan Cotterell (ACL (1), 2023)

A Length-Extrapolatable Transformer. - Yutao Sun; Li Dong 004; Barun Patra; Shuming Ma; Shaohan Huang; Alon Benhaim; Vishrav Chaudhary; Xia Song; Furu Wei (ACL (1), 2023)

GanLM: Encoder-Decoder Pre-training with an Auxiliary Discriminator. - Jian Yang 030; Shuming Ma; Li Dong 004; Shaohan Huang; Haoyang Huang; Yuwei Yin; Dongdong Zhang 001; Liqun Yang; Furu Wei; Zhoujun Li 001 (ACL (1), 2023)

HanoiT: Enhancing Context-aware Translation via Selective Context. - Jian Yang 030; Yuwei Yin; Shuming Ma; Liqun Yang; Hongcheng Guo; Haoyang Huang; Dongdong Zhang 001; Yutao Zeng; Zhoujun Li 001; Furu Wei (DASFAA (3), 2023)

TRIP: Accelerating Document-level Multilingual Pre-training via Triangular Document-level Pre-training on Parallel Data Triplets. - Hongyuan Lu; Haoyang Huang; Shuming Ma; Dongdong Zhang 001; Wai Lam; Zhaochuan Gao; Anthony Aue; Arul Menezes; Furu Wei (EMNLP (Findings), 2023)

Are More Layers Beneficial to Graph Transformers? - Haiteng Zhao; Shuming Ma; Dongdong Zhang 001; Zhi-Hong Deng 001; Furu Wei (ICLR, 2023)

Magneto: A Foundation Transformer. - Hongyu Wang 009; Shuming Ma; Shaohan Huang; Li Dong 004; Wenhui Wang 003; Zhiliang Peng; Yu Wu; Payal Bajaj; Saksham Singhal; Alon Benhaim; Barun Patra; Zhun Liu; et al. (ICML, 2023)

On the Pareto Front of Multilingual Neural Machine Translation. - Liang Chen 024; Shuming Ma; Dongdong Zhang 001; Furu Wei; Baobao Chang (NeurIPS, 2023)

Language Is Not All You Need: Aligning Perception with Language Models. - Shaohan Huang; Li Dong 004; Wenhui Wang 003; Yaru Hao; Saksham Singhal; Shuming Ma; Tengchao Lv; Lei Cui 001; Owais Khan Mohammed; Barun Patra; Qiang Liu; Kriti Aggarwal; et al. (NeurIPS, 2023)

Discourse Centric Evaluation of Machine Translation with a Densely Annotated Parallel Corpus. - Yuchen Eleanor Jiang; Tianyu Liu 004; Shuming Ma; Dongdong Zhang 001; Mrinmaya Sachan; Ryan Cotterell (CoRR, 2023)

Kosmos-2: Grounding Multimodal Large Language Models to the World. - Zhiliang Peng; Wenhui Wang 003; Li Dong 004; Yaru Hao; Shaohan Huang; Shuming Ma; Furu Wei (CoRR, 2023)

LongNet: Scaling Transformers to 1, 000, 000, 000 Tokens. - Jiayu Ding; Shuming Ma; Li Dong 004; Xingxing Zhang 002; Shaohan Huang; Wenhui Wang 003; Nanning Zheng 001; Furu Wei (CoRR, 2023)

Retentive Network: A Successor to Transformer for Large Language Models. - Yutao Sun; Li Dong 004; Shaohan Huang; Shuming Ma; Yuqing Xia; Jilong Xue; Jianyong Wang 001; Furu Wei (CoRR, 2023)

Kosmos-2.5: A Multimodal Literate Model. - Tengchao Lv; Yupan Huang; Jingye Chen; Lei Cui 001; Shuming Ma; Yaoyao Chang; Shaohan Huang; Wenhui Wang 003; Li Dong 004; Weiyao Luo; Shaoxiang Wu; Guoxin Wang; et al. (CoRR, 2023)

BitNet: Scaling 1-bit Transformers for Large Language Models. - Hongyu Wang 009; Shuming Ma; Li Dong 004; Shaohan Huang; Huaijie Wang; Lingxiao Ma; Fan Yang 024; Ruiping Wang; Yi Wu 013; Furu Wei (CoRR, 2023)

Auto-ICL: In-Context Learning without Human Supervision. - Jinghan Yang; Shuming Ma; Furu Wei (CoRR, 2023)

When an Image is Worth 1, 024 x 1, 024 Words: A Case Study in Computational Pathology. - Wenhui Wang 003; Shuming Ma; Hanwen Xu; Naoto Usuyama; Jiayu Ding; Hoifung Poon; Furu Wei (CoRR, 2023)

GTrans: Grouping and Fusing Transformer Layers for Neural Machine Translation. - Jian Yang 030; Yuwei Yin; Liqun Yang; Shuming Ma; Haoyang Huang; Dongdong Zhang 001; Furu Wei; Zhoujun Li 001 (IEEE ACM Trans. Audio Speech Lang. Process., 2023)

Grounding Multimodal Large Language Models to the World. - Zhiliang Peng; Wenhui Wang 003; Li Dong 004; Yaru Hao; Shaohan Huang; Shuming Ma; Qixiang Ye; Furu Wei (ICLR, 2024)

KOSMOS-E : Learning to Follow Instruction for Robotic Grasping. - Zhi Wang; Xun Wu; Shaohan Huang; Li Dong 004; Wenhui Wang 003; Shuming Ma; Furu Wei (IROS, 2024)

You Only Cache Once: Decoder-Decoder Architectures for Language Models. - Yutao Sun; Li Dong 004; Yi Zhu; Shaohan Huang; Wenhui Wang 003; Shuming Ma; Quanlu Zhang; Jianyong Wang 001; Furu Wei (NeurIPS, 2024)

Multi-Head Mixture-of-Experts. - Xun Wu; Shaohan Huang; Wenhui Wang 003; Shuming Ma; Li Dong 004; Furu Wei (NeurIPS, 2024)

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits. - Shuming Ma; Hongyu Wang 009; Lingxiao Ma; Lei Wang 222; Wenhui Wang 003; Shaohan Huang; Li Dong 004; Ruiping Wang; Jilong Xue; Furu Wei (CoRR, 2024)

Q-Sparse: All Large Language Models can be Fully Sparsely-Activated. - Hongyu Wang 009; Shuming Ma; Ruiping Wang; Furu Wei (CoRR, 2024)

1-bit AI Infra: Part 1.1, Fast and Lossless BitNet b1.58 Inference on CPUs. - Jinheng Wang; Hansong Zhou; Ting Song; Shaoguang Mao; Shuming Ma; Hongyu Wang; Yan Xia 005; Furu Wei (CoRR, 2024)

BitNet a4.8: 4-bit Activations for 1-bit LLMs. - Hongyu Wang; Shuming Ma; Furu Wei (CoRR, 2024)

MH-MoE: Multi-Head Mixture-of-Experts. - Shaohan Huang; Xun Wu; Shuming Ma; Furu Wei (CoRR, 2024)

RedStone: Curating General, Code, Math, and QA Data for Large Language Models. - Yaoyao Chang; Lei Cui 001; Li Dong 004; Shaohan Huang; Yangyu Huang; Yupan Huang; Scarlett Li; Tengchao Lv; Shuming Ma; Qinzheng Sun; Wenhui Wang 003; Furu Wei; et al. (CoRR, 2024)

Integrating fuzzy analytic hierarchy process into ecosystem service-based spatial planning: A case study of the Shenyang metropolitan area, China. - Yunkai Fan; Shuming Ma (Ecol. Informatics, 2024)

DeepNet: Scaling Transformers to 1,000 Layers. - Hongyu Wang 009; Shuming Ma; Li Dong 004; Shaohan Huang; Dongdong Zhang 001; Furu Wei (IEEE Trans. Pattern Anal. Mach. Intell., 2024)

Bitnet.cpp: Efficient Edge Inference for Ternary LLMs. - Jinheng Wang; Hansong Zhou; Ting Song; Shijie Cao; Yan Xia 005; Ting Cao; Jianyu Wei; Shuming Ma; Hongyu Wang; Furu Wei (ACL (1), 2025)

Next Block Prediction: Video Generation via Semi-Autoregressive Modeling. - Shuhuai Ren; Shuming Ma; Xu Sun 001; Furu Wei (CoRR, 2025)

Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoning. - Wenkai Yang; Shuming Ma; Yankai Lin; Furu Wei (CoRR, 2025)

BitNet b1.58 2B4T Technical Report. - Shuming Ma; Hongyu Wang; Shaohan Huang; Xingxing Zhang 002; Ying Hu; Ting Song; Yan Xia 005; Furu Wei (CoRR, 2025)

BitNet v2: Native 4-bit Activations with Hadamard Transformation for 1-bit LLMs. - Hongyu Wang 009; Shuming Ma; Furu Wei (CoRR, 2025)