ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning. - Vamsi Aribandi; Yi Tay; Tal Schuster; Jinfeng Rao; Huaixiu Steven Zheng; Sanket Vaibhav Mehta; Honglei Zhuang; Vinh Q. Tran 002; Dara Bahri; Jianmo Ni; Jai Prakash Gupta 001; Kai Hui 001; et al. (ICLR, 2022)

HyperPrompt: Prompt-based Task-Conditioning of Transformers. - Yun He; Huaixiu Steven Zheng; Yi Tay; Jai Prakash Gupta 001; Yu Du; Vamsi Aribandi; Zhe Zhao 001; YaGuang Li; Zhao Chen; Donald Metzler; Heng-Tze Cheng; Ed H. Chi (ICML, 2022)

LaMDA: Language Models for Dialog Applications. - Romal Thoppilan; Daniel De Freitas; Jamie Hall; Noam Shazeer; Apoorv Kulshreshtha; Heng-Tze Cheng; Alicia Jin; Taylor Bos; Leslie Baker; Yu Du; YaGuang Li; Hongrae Lee; et al. (CoRR, 2022)

Unifying Language Learning Paradigms. - Yi Tay; Mostafa Dehghani 001; Vinh Q. Tran 002; Xavier Garcia; Dara Bahri; Tal Schuster; Huaixiu Steven Zheng; Neil Houlsby; Donald Metzler (CoRR, 2022)

Transcending Scaling Laws with 0.1% Extra Compute. - Yi Tay; Jason Wei; Hyung Won Chung; Vinh Q. Tran 002; David R. So; Siamak Shakeri; Xavier Garcia; Huaixiu Steven Zheng; Jinfeng Rao; Aakanksha Chowdhery; Denny Zhou; Donald Metzler; et al. (EMNLP, 2023)

UL2: Unifying Language Learning Paradigms. - Yi Tay; Mostafa Dehghani 001; Vinh Q. Tran 002; Xavier Garcia; Jason Wei; Xuezhi Wang 002; Hyung Won Chung; Dara Bahri; Tal Schuster; Huaixiu Steven Zheng; Denny Zhou; Neil Houlsby; et al. (ICLR, 2023)

Large Language Models Cannot Self-Correct Reasoning Yet. - Jie Huang 009; Xinyun Chen; Swaroop Mishra; Huaixiu Steven Zheng; Adams Wei Yu; Xinying Song; Denny Zhou (ICLR, 2024)

Take a Step Back: Evoking Reasoning via Abstraction in Large Language Models. - Huaixiu Steven Zheng; Swaroop Mishra; Xinyun Chen; Heng-Tze Cheng; Ed H. Chi; Quoc V. Le; Denny Zhou (ICLR, 2024)

SELF-DISCOVER: Large Language Models Self-Compose Reasoning Structures. - Pei Zhou; Jay Pujara; Xiang Ren 001; Xinyun Chen; Heng-Tze Cheng; Quoc V. Le; Ed H. Chi; Denny Zhou; Swaroop Mishra; Huaixiu Steven Zheng (NeurIPS, 2024)

NATURAL PLAN: Benchmarking LLMs on Natural Language Planning. - Huaixiu Steven Zheng; Swaroop Mishra; Hugh Zhang; Xinyun Chen; Minmin Chen; Azade Nova; Le Hou; Heng-Tze Cheng; Quoc V. Le; Ed H. Chi; Denny Zhou (CoRR, 2024)

Speculative RAG: Enhancing Retrieval Augmented Generation through Drafting. - Zilong Wang 002; Zifeng Wang 002; Long T. Le; Huaixiu Steven Zheng; Swaroop Mishra; Vincent Perot; Yuwei Zhang 001; Anush Mattapalli; Ankur Taly; Jingbo Shang; Chen-Yu Lee; Tomas Pfister (ICLR, 2025)