Double or Nothing: Multiplicative Incentive Mechanisms for Crowdsourcing. - Nihar Bhadresh Shah; Denny Zhou (NIPS, 2015)

Neural Logic Machines. - Honghua Dong; Jiayuan Mao; Tian Lin; Chong Wang 002; Lihong Li 001; Denny Zhou (ICLR (Poster), 2019)

Doubly Sparse: Sparse Mixture of Sparse Experts for Efficient Softmax Inference. - Shun Liao; Ting Chen; Tian Lin; Denny Zhou; Chong Wang (CoRR, 2019)

Extreme Language Model Compression with Optimal Subwords and Shared Projections. - Sanqiang Zhao; Raghav Gupta; Yang Song; Denny Zhou (CoRR, 2019)

Deep Physiological State Space Model for Clinical Forecasting. - Yuan Xue; Denny Zhou; Nan Du 002; Andrew M. Dai; Zhen Xu 006; Kun Zhang 043; Claire Cui (CoRR, 2019)

MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices. - Zhiqing Sun; Hongkun Yu 001; Xiaodan Song; Renjie Liu; Yiming Yang; Denny Zhou (ACL, 2020)

Neural Symbolic Reader: Scalable Integration of Distributed and Symbolic Representations for Reading Comprehension. - Xinyun Chen; Chen Liang; Adams Wei Yu; Denny Zhou; Dawn Song; Quoc V. Le (ICLR, 2020)

Black-box Off-policy Estimation for Infinite-Horizon Reinforcement Learning. - Ali Mousavi 003; Lihong Li 001; Qiang Liu 001; Denny Zhou (ICLR, 2020)

Good Subnetworks Provably Exist: Pruning via Greedy Forward Selection. - Mao Ye 006; Chengyue Gong; Lizhen Nie; Denny Zhou; Adam R. Klivans; Qiang Liu 001 (ICML, 2020)

Go Wide, Then Narrow: Efficient Training of Deep Thin Networks. - Denny Zhou; Mao Ye 006; Chen Chen; Tianjian Meng; Mingxing Tan; Xiaodan Song; Quoc V. Le; Qiang Liu 001; Dale Schuurmans (ICML, 2020)

Deep State-Space Generative Model For Correlated Time-to-Event Predictions. - Yuan Xue; Denny Zhou; Nan Du 002; Andrew M. Dai; Zhen Xu 006; Kun Zhang 043; Claire Cui (KDD, 2020)

Compositional Generalization via Neural-Symbolic Stack Machines. - Xinyun Chen; Chen Liang; Adams Wei Yu; Dawn Song; Denny Zhou (NeurIPS, 2020)

Linear-Time WordPiece Tokenization. - Xinying Song; Alex Salcianu; Yang Song; Dave Dopson; Denny Zhou (CoRR, 2020)

Extremely Small BERT Models from Mixed-Vocabulary Training. - Sanqiang Zhao; Raghav Gupta; Yang Song; Denny Zhou (EACL, 2021)

Fast WordPiece Tokenization. - Xinying Song; Alex Salcianu; Yang Song; Dave Dopson; Denny Zhou (EMNLP (1), 2021)

SpreadsheetCoder: Formula Prediction from Semi-structured Context. - Xinyun Chen; Petros Maniatis; Rishabh Singh; Charles Sutton; Hanjun Dai; Max Lin; Denny Zhou (ICML, 2021)

LEGO: Latent Execution-Guided Reasoning for Multi-Hop Question Answering on Knowledge Graphs. - Hongyu Ren; Hanjun Dai; Bo Dai 001; Xinyun Chen; Michihiro Yasunaga; Haitian Sun; Dale Schuurmans; Jure Leskovec; Denny Zhou (ICML, 2021)

Speeding up Deep Model Training by Sharing Weights and Then Unsharing. - Shuo Yang; Le Hou; Xiaodan Song; Qiang Liu 001; Denny Zhou (CoRR, 2021)

A Simple Single-Scale Vision Transformer for Object Localization and Instance Segmentation. - Wuyang Chen 001; Xianzhi Du; Fan Yang; Lucas Beyer; Xiaohua Zhai; Tsung-Yi Lin; Huizhong Chen; Jing Li; Xiaodan Song; Zhangyang Wang; Denny Zhou (CoRR, 2021)

Token Dropping for Efficient BERT Pretraining. - Le Hou; Richard Yuanzhe Pang; Tianyi Zhou 001; Yuexin Wu; Xinying Song; Xiaodan Song; Denny Zhou (ACL (1), 2022)

DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detection. - Yingwei Li 002; Adams Wei Yu; Tianjian Meng; Benjamin Caine; Jiquan Ngiam; Daiyi Peng; Junyang Shen; Yifeng Lu; Denny Zhou; Quoc V. Le; Alan L. Yuille; Mingxing Tan (CVPR, 2022)

A Simple Single-Scale Vision Transformer for Object Detection and Instance Segmentation. - Wuyang Chen 001; Xianzhi Du; Fan Yang; Lucas Beyer; Xiaohua Zhai; Tsung-Yi Lin; Huizhong Chen; Jing Li; Xiaodan Song; Zhangyang Wang; Denny Zhou (ECCV (10), 2022)

Auto-scaling Vision Transformers without Training. - Wuyang Chen 001; Wei Huang 034; Xianzhi Du; Xiaodan Song; Zhangyang Wang; Denny Zhou (ICLR, 2022)

Provable Stochastic Optimization for Global Contrastive Learning: Small Batch Does Not Harm Performance. - Zhuoning Yuan; Yuexin Wu; Zi-Hao Qiu; Xianzhi Du; Lijun Zhang 005; Denny Zhou; Tianbao Yang (ICML, 2022)

SMORE: Knowledge Graph Completion and Multi-hop Reasoning in Massive Knowledge Graphs. - Hongyu Ren; Hanjun Dai; Bo Dai 001; Xinyun Chen; Denny Zhou; Jure Leskovec; Dale Schuurmans (KDD, 2022)

Back Razor: Memory-Efficient Transfer Learning by Self-Sparsified Backpropagation. - Ziyu Jiang; Xuxi Chen; Xueqin Huang; Xianzhi Du; Denny Zhou; Zhangyang Wang (NeurIPS, 2022)

Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. - Jason Wei; Xuezhi Wang 002; Dale Schuurmans; Maarten Bosma; Brian Ichter; Fei Xia 002; Ed H. Chi; Quoc V. Le; Denny Zhou (NeurIPS, 2022)

Chain of Thought Prompting Elicits Reasoning in Large Language Models. - Jason Wei; Xuezhi Wang 002; Dale Schuurmans; Maarten Bosma; Ed H. Chi; Quoc Le 001; Denny Zhou (CoRR, 2022)

DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detection. - Yingwei Li 002; Adams Wei Yu; Tianjian Meng; Benjamin Caine; Jiquan Ngiam; Daiyi Peng; Junyang Shen; Bo Wu; Yifeng Lu; Denny Zhou; Quoc V. Le; Alan L. Yuille; et al. (CoRR, 2022)

Self-Consistency Improves Chain of Thought Reasoning in Language Models. - Xuezhi Wang 002; Jason Wei; Dale Schuurmans; Quoc V. Le; Ed H. Chi; Denny Zhou (CoRR, 2022)

Least-to-Most Prompting Enables Complex Reasoning in Large Language Models. - Denny Zhou; Nathanael Schärli; Le Hou; Jason Wei; Nathan Scales; Xuezhi Wang 002; Dale Schuurmans; Olivier Bousquet; Quoc Le 001; Ed H. Chi (CoRR, 2022)

Rationale-Augmented Ensembles in Language Models. - Xuezhi Wang 002; Jason Wei; Dale Schuurmans; Quoc V. Le; Ed H. Chi; Denny Zhou (CoRR, 2022)

Scaling Instruction-Finetuned Language Models. - Hyung Won Chung; Le Hou; Shayne Longpre; Barret Zoph; Yi Tay; William Fedus; Eric Li; Xuezhi Wang 002; Mostafa Dehghani 001; Siddhartha Brahma; Albert Webson; Shixiang Shane Gu; et al. (CoRR, 2022)

TEMPERA: Test-Time Prompting via Reinforcement Learning. - Tianjun Zhang; Xuezhi Wang 002; Denny Zhou; Dale Schuurmans; Joseph E. Gonzalez (CoRR, 2022)

Emergent Abilities of Large Language Models. - Jason Wei; Yi Tay; Rishi Bommasani; Colin Raffel; Barret Zoph; Sebastian Borgeaud; Dani Yogatama; Maarten Bosma; Denny Zhou; Donald Metzler; Ed H. Chi; Tatsunori Hashimoto; et al. (Trans. Mach. Learn. Res., 2022)

Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them. - Mirac Suzgun; Nathan Scales; Nathanael Schärli; Sebastian Gehrmann; Yi Tay; Hyung Won Chung; Aakanksha Chowdhery; Quoc V. Le; Ed H. Chi; Denny Zhou; Jason Wei (ACL (Findings), 2023)

Transcending Scaling Laws with 0.1% Extra Compute. - Yi Tay; Jason Wei; Hyung Won Chung; Vinh Q. Tran 002; David R. So; Siamak Shakeri; Xavier Garcia; Huaixiu Steven Zheng; Jinfeng Rao; Aakanksha Chowdhery; Denny Zhou; Donald Metzler; et al. (EMNLP, 2023)

Symbol tuning improves in-context learning in language models. - Jerry W. Wei; Le Hou; Andrew K. Lampinen; Xiangning Chen; Da Huang; Yi Tay; Xinyun Chen; Yifeng Lu; Denny Zhou; Tengyu Ma 001; Quoc V. Le (EMNLP, 2023)

Self-Consistency Improves Chain of Thought Reasoning in Language Models. - Xuezhi Wang 002; Jason Wei; Dale Schuurmans; Quoc V. Le; Ed H. Chi; Sharan Narang; Aakanksha Chowdhery; Denny Zhou (ICLR, 2023)

What learning algorithm is in-context learning? Investigations with linear models. - Ekin Akyürek; Dale Schuurmans; Jacob Andreas; Tengyu Ma 001; Denny Zhou (ICLR, 2023)

Compositional Semantic Parsing with Large Language Models. - Andrew Drozdov; Nathanael Schärli; Ekin Akyürek; Nathan Scales; Xinying Song; Xinyun Chen; Olivier Bousquet; Denny Zhou (ICLR, 2023)

Mind's Eye: Grounded Language Model Reasoning through Simulation. - Ruibo Liu; Jason Wei; Shixiang Shane Gu; Te-Yen Wu; Soroush Vosoughi; Claire Cui; Denny Zhou; Andrew M. Dai (ICLR, 2023)

Language models are multilingual chain-of-thought reasoners. - Freda Shi; Mirac Suzgun; Markus Freitag; Xuezhi Wang 002; Suraj Srivats; Soroush Vosoughi; Hyung Won Chung; Yi Tay; Sebastian Ruder; Denny Zhou; Dipanjan Das 001; Jason Wei (ICLR, 2023)

Recitation-Augmented Language Models. - Zhiqing Sun; Xuezhi Wang 002; Yi Tay; Yiming Yang; Denny Zhou (ICLR, 2023)

UL2: Unifying Language Learning Paradigms. - Yi Tay; Mostafa Dehghani 001; Vinh Q. Tran 002; Xavier Garcia; Jason Wei; Xuezhi Wang 002; Hyung Won Chung; Dara Bahri; Tal Schuster; Huaixiu Steven Zheng; Denny Zhou; Neil Houlsby; et al. (ICLR, 2023)

TEMPERA: Test-Time Prompt Editing via Reinforcement Learning. - Tianjun Zhang; Xuezhi Wang 002; Denny Zhou; Dale Schuurmans; Joseph E. Gonzalez (ICLR, 2023)

Least-to-Most Prompting Enables Complex Reasoning in Large Language Models. - Denny Zhou; Nathanael Schärli; Le Hou; Jason Wei; Nathan Scales; Xuezhi Wang 002; Dale Schuurmans; Claire Cui; Olivier Bousquet; Quoc V. Le; Ed H. Chi (ICLR, 2023)

The Flan Collection: Designing Data and Methods for Effective Instruction Tuning. - Shayne Longpre; Le Hou; Tu Vu; Albert Webson; Hyung Won Chung; Yi Tay; Denny Zhou; Quoc V. Le; Barret Zoph; Jason Wei; Adam Roberts (ICML, 2023)

Not All Semantics are Created Equal: Contrastive Self-supervised Learning with Automatic Temperature Individualization. - Zi-Hao Qiu; Quanqi Hu; Zhuoning Yuan; Denny Zhou; Lijun Zhang 005; Tianbao Yang (ICML, 2023)

Large Language Models Can Be Easily Distracted by Irrelevant Context. - Freda Shi; Xinyun Chen; Kanishka Misra; Nathan Scales; David Dohan; Ed H. Chi; Nathanael Schärli; Denny Zhou (ICML, 2023)

Larger language models do in-context learning differently. - Jerry W. Wei; Jason Wei; Yi Tay; Dustin Tran; Albert Webson; Yifeng Lu; Xinyun Chen; Hanxiao Liu; Da Huang; Denny Zhou; Tengyu Ma 001 (CoRR, 2023)

Flan-MoE: Scaling Instruction-Finetuned Language Models with Sparse Mixture of Experts. - Sheng Shen 001; Le Hou; Yanqi Zhou; Nan Du 002; Shayne Longpre; Jason Wei; Hyung Won Chung; Barret Zoph; William Fedus; Xinyun Chen; Tu Vu; Yuexin Wu; et al. (CoRR, 2023)

Training Socially Aligned Language Models in Simulated Human Society. - Ruibo Liu; Ruixin Yang; Chenyan Jia; Ge Zhang; Denny Zhou; Andrew M. Dai; Diyi Yang; Soroush Vosoughi (CoRR, 2023)

Simple synthetic data reduces sycophancy in large language models. - Jerry W. Wei; Da Huang; Yifeng Lu; Denny Zhou; Quoc V. Le (CoRR, 2023)

Large Language Models can Learn Rules. - Zhaocheng Zhu; Yuan Xue; Xinyun Chen; Denny Zhou; Jian Tang 005; Dale Schuurmans; Hanjun Dai (CoRR, 2023)

Instruction-Following Evaluation for Large Language Models. - Jeffrey Zhou; Tianjian Lu; Swaroop Mishra; Siddhartha Brahma; Sujoy Basu; Yi Luan; Denny Zhou; Le Hou (CoRR, 2023)

Universal Self-Consistency for Large Language Model Generation. - Xinyun Chen; Renat Aksitov; Uri Alon 002; Jie Ren; Kefan Xiao; Pengcheng Yin; Sushant Prakash; Charles Sutton; Xuezhi Wang 002; Denny Zhou (CoRR, 2023)

PaLM: Scaling Language Modeling with Pathways. - Aakanksha Chowdhery; Sharan Narang; Jacob Devlin; Maarten Bosma; Gaurav Mishra; Adam Roberts; Paul Barham 001; Hyung Won Chung; Charles Sutton; Sebastian Gehrmann; Parker Schuh; Kensen Shi; et al. (J. Mach. Learn. Res., 2023)

FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation. - Tu Vu; Mohit Iyyer; Xuezhi Wang 002; Noah Constant; Jerry W. Wei; Jason Wei; Chris Tar; Yun-Hsuan Sung; Denny Zhou; Quoc V. Le; Thang Luong (ACL (Findings), 2024)

Chain of Thought Empowers Transformers to Solve Inherently Serial Problems. - Zhiyuan Liu 001; Hong Liu; Denny Zhou; Tengyu Ma 001 (ICLR, 2024)

Large Language Models Cannot Self-Correct Reasoning Yet. - Jie Huang 009; Xinyun Chen; Swaroop Mishra; Huaixiu Steven Zheng; Adams Wei Yu; Xinying Song; Denny Zhou (ICLR, 2024)

Large Language Models as Tool Makers. - Tianle Cai; Xuezhi Wang 002; Tengyu Ma 001; Xinyun Chen; Denny Zhou (ICLR, 2024)

Teaching Large Language Models to Self-Debug. - Xinyun Chen; Maxwell Lin; Nathanael Schärli; Denny Zhou (ICLR, 2024)

Mixture-of-Experts Meets Instruction Tuning: A Winning Combination for Large Language Models. - Sheng Shen 001; Le Hou; Yanqi Zhou; Nan Du 002; Shayne Longpre; Jason Wei; Hyung Won Chung; Barret Zoph; William Fedus; Xinyun Chen; Tu Vu; Yuexin Wu; et al. (ICLR, 2024)

Large Language Models as Optimizers. - Chengrun Yang; Xuezhi Wang 002; Yifeng Lu; Hanxiao Liu; Quoc V. Le; Denny Zhou; Xinyun Chen (ICLR, 2024)

Large Language Models as Analogical Reasoners. - Michihiro Yasunaga; Xinyun Chen; Yujia Li; Panupong Pasupat; Jure Leskovec; Percy Liang; Ed H. Chi; Denny Zhou (ICLR, 2024)

Take a Step Back: Evoking Reasoning via Abstraction in Large Language Models. - Huaixiu Steven Zheng; Swaroop Mishra; Xinyun Chen; Heng-Tze Cheng; Ed H. Chi; Quoc V. Le; Denny Zhou (ICLR, 2024)

Premise Order Matters in Reasoning with Large Language Models. - Xinyun Chen; Ryan A. Chi; Xuezhi Wang 002; Denny Zhou (ICML, 2024)

A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicity. - Shayne Longpre; Gregory Yauney; Emily Reif; Katherine Lee; Adam Roberts; Barret Zoph; Denny Zhou; Jason Wei; Kevin Robinson; David Mimno; Daphne Ippolito (NAACL-HLT, 2024)

Chain-of-Thought Reasoning Without Prompting. - Xuezhi Wang 002; Denny Zhou (NeurIPS, 2024)

SELF-DISCOVER: Large Language Models Self-Compose Reasoning Structures. - Pei Zhou; Jay Pujara; Xiang Ren 001; Xinyun Chen; Heng-Tze Cheng; Quoc V. Le; Ed H. Chi; Denny Zhou; Swaroop Mishra; Huaixiu Steven Zheng (NeurIPS, 2024)

Transformers Can Achieve Length Generalization But Not Robustly. - Yongchao Zhou; Uri Alon 002; Xinyun Chen; Xuezhi Wang 002; Rishabh Agarwal; Denny Zhou (CoRR, 2024)

Chain of Thought Empowers Transformers to Solve Inherently Serial Problems. - Zhiyuan Li 005; Hong Liu; Denny Zhou; Tengyu Ma 001 (CoRR, 2024)

Best Practices and Lessons Learned on Synthetic Data for Language Models. - Ruibo Liu; Jerry Wei; Fangyu Liu; Chenglei Si; Yanzhe Zhang; Jinmeng Rao; Steven Zheng; Daiyi Peng; Diyi Yang; Denny Zhou; Andrew M. Dai (CoRR, 2024)

CRISPR-GPT: An LLM Agent for Automated Design of Gene-Editing Experiments. - Kaixuan Huang; Yuanhao Qu; Henry Cousins; William A. Johnson; Di Yin; Mihir Shah; Denny Zhou; Russ B. Altman; Mengdi Wang; Le Cong (CoRR, 2024)

NATURAL PLAN: Benchmarking LLMs on Natural Language Planning. - Huaixiu Steven Zheng; Swaroop Mishra; Hugh Zhang; Xinyun Chen; Minmin Chen; Azade Nova; Le Hou; Heng-Tze Cheng; Quoc V. Le; Ed H. Chi; Denny Zhou (CoRR, 2024)

Scaling Instruction-Finetuned Language Models. - Hyung Won Chung; Le Hou; Shayne Longpre; Barret Zoph; Yi Tay; William Fedus; Yunxuan Li; Xuezhi Wang 002; Mostafa Dehghani 001; Siddhartha Brahma; Albert Webson; Shixiang Shane Gu; et al. (J. Mach. Learn. Res., 2024)