William Fedus

According to our database, William Fedus authored at least 26 papers between 2018 and 2023.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of four.

2023

PaLM: Scaling Language Modeling with Pathways.

Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengcheng Yin, Toju Duke, Anselm Levskaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier Garcia, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito, David Luan, Hyeontaek Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, Andrew M. Dai, Thanumalayan Sankaranarayana Pillai, Marie Pellat, Aitor Lewkowycz, Erica Moreira, Rewon Child, Oleksandr Polozov, Katherine Lee, Zongwei Zhou, Xuezhi Wang, Brennan Saeta, Mark Diaz, Orhan Firat, Michele Catasta, Jason Wei, Kathy Meier-Hellstern, Douglas Eck, Jeff Dean, Slav Petrov, Noah Fiedel

J. Mach. Learn. Res., 2023

Flan-MoE: Scaling Instruction-Finetuned Language Models with Sparse Mixture of Experts.

Sheng Shen, Le Hou, Yanqi Zhou, Nan Du, Shayne Longpre, Jason Wei, Hyung Won Chung, Barret Zoph, William Fedus, Xinyun Chen, Tu Vu, Yuexin Wu, Wuyang Chen, Albert Webson, Yunxuan Li, Vincent Y. Zhao, Hongkun Yu, Kurt Keutzer, Trevor Darrell, Denny Zhou

CoRR, 2023

Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling?

Yi Tay, Mostafa Dehghani, Samira Abnar, Hyung Won Chung, William Fedus, Jinfeng Rao, Sharan Narang, Vinh Q. Tran, Dani Yogatama, Donald Metzler

Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2023, 2023

2022

Emergent Abilities of Large Language Models.

Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, William Fedus

Trans. Mach. Learn. Res., 2022

Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity.

William Fedus, Barret Zoph, Noam Shazeer

J. Mach. Learn. Res., 2022

Scaling Instruction-Finetuned Language Models.

Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Y. Zhao, Yanping Huang, Andrew M. Dai, Hongkun Yu, Slav Petrov, Ed H. Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoc V. Le, Jason Wei

CoRR, 2022

A Review of Sparse Expert Models in Deep Learning.

William Fedus, Jeff Dean, Barret Zoph

CoRR, 2022

Designing Effective Sparse Expert Models.

Barret Zoph, Irwan Bello, Sameer Kumar, Nan Du, Yanping Huang, Jeff Dean, Noam Shazeer, William Fedus

CoRR, 2022

GLaM: Efficient Scaling of Language Models with Mixture-of-Experts.

Nan Du, Yanping Huang, Andrew M. Dai, Simon Tong, Dmitry Lepikhin, Yuanzhong Xu, Maxim Krikun, Yanqi Zhou, Adams Wei Yu, Orhan Firat, Barret Zoph, Liam Fedus, Maarten P. Bosma, Zongwei Zhou, Tao Wang, Yu Emma Wang, Kellie Webster, Marie Pellat, Kevin Robinson, Kathleen S. Meier-Hellstern, Toju Duke, Lucas Dixon, Kun Zhang, Quoc V. Le, Yonghui Wu, Zhifeng Chen, Claire Cui

Proceedings of the International Conference on Machine Learning, 2022

Scale Efficiently: Insights from Pretraining and Finetuning Transformers.

Yi Tay, Mostafa Dehghani, Jinfeng Rao, William Fedus, Samira Abnar, Hyung Won Chung, Sharan Narang, Dani Yogatama, Ashish Vaswani, Donald Metzler

Proceedings of the Tenth International Conference on Learning Representations, 2022

2021

Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers.

Yi Tay, Mostafa Dehghani, Jinfeng Rao, William Fedus, Samira Abnar, Hyung Won Chung, Sharan Narang, Dani Yogatama, Ashish Vaswani, Donald Metzler

CoRR, 2021

Revisiting ResNets: Improved Training and Scaling Strategies.

Irwan Bello, William Fedus, Xianzhi Du, Ekin Dogus Cubuk, Aravind Srinivas, Tsung-Yi Lin, Jonathon Shlens, Barret Zoph

Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, 2021

Do Transformer Modifications Transfer Across Implementations and Applications?

Sharan Narang, Hyung Won Chung, Yi Tay, Liam Fedus, Thibault Févry, Michael Matena, Karishma Malkan, Noah Fiedel, Noam Shazeer, Zhenzhong Lan, Yanqi Zhou, Wei Li, Nan Ding, Jake Marcus, Adam Roberts, Colin Raffel

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021

2020

On Catastrophic Interference in Atari 2600 Games.

William Fedus, Dibya Ghosh, John D. Martin, Marc G. Bellemare, Yoshua Bengio, Hugo Larochelle

CoRR, 2020

Revisiting Fundamentals of Experience Replay.

William Fedus, Prajit Ramachandran, Rishabh Agarwal, Yoshua Bengio, Hugo Larochelle, Mark Rowland, Will Dabney

Proceedings of the 37th International Conference on Machine Learning, 2020

On Bonus Based Exploration Methods In The Arcade Learning Environment.

Adrien Ali Taïga, William Fedus, Marlos C. Machado, Aaron C. Courville, Marc G. Bellemare

Proceedings of the 8th International Conference on Learning Representations, 2020

Language GANs Falling Short.

Massimo Caccia, Lucas Caccia, William Fedus, Hugo Larochelle, Joelle Pineau, Laurent Charlin

Proceedings of the 8th International Conference on Learning Representations, 2020

Algorithmic Improvements for Deep Reinforcement Learning Applied to Interactive Fiction.

Vishal Jain, William Fedus, Hugo Larochelle, Doina Precup, Marc G. Bellemare

Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence, 2020

2019

Benchmarking Bonus-Based Exploration Methods on the Arcade Learning Environment.

Adrien Ali Taïga, William Fedus, Marlos C. Machado, Aaron C. Courville, Marc G. Bellemare

CoRR, 2019

Hyperbolic Discounting and Learning over Multiple Horizons.

William Fedus, Carles Gelada, Yoshua Bengio, Marc G. Bellemare, Hugo Larochelle

CoRR, 2019

Deep Graph Infomax.

Petar Velickovic, William Fedus, William L. Hamilton, Pietro Liò, Yoshua Bengio, R. Devon Hjelm

Proceedings of the 7th International Conference on Learning Representations, 2019

Recall Traces: Backtracking Models for Efficient Reinforcement Learning.

Anirudh Goyal, Philemon Brakel, William Fedus, Soumye Singhal, Timothy P. Lillicrap, Sergey Levine, Hugo Larochelle, Yoshua Bengio

Proceedings of the 7th International Conference on Learning Representations, 2019

2018

Recall Traces: Backtracking Models for Efficient Reinforcement Learning.

Anirudh Goyal, Philemon Brakel, William Fedus, Timothy P. Lillicrap, Sergey Levine, Hugo Larochelle, Yoshua Bengio

CoRR, 2018

Disentangling the independently controllable factors of variation by interacting with the world.

Valentin Thomas, Emmanuel Bengio, William Fedus, Jules Pondard, Philippe Beaudoin, Hugo Larochelle, Joelle Pineau, Doina Precup, Yoshua Bengio

CoRR, 2018

Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step.

William Fedus, Mihaela Rosca, Balaji Lakshminarayanan, Andrew M. Dai, Shakir Mohamed, Ian J. Goodfellow

Proceedings of the 6th International Conference on Learning Representations, 2018

MaskGAN: Better Text Generation via Filling in the _______.

William Fedus, Ian J. Goodfellow, Andrew M. Dai

Proceedings of the 6th International Conference on Learning Representations, 2018