Noam Shazeer

According to our database, Noam Shazeer authored at least 49 papers between 1999 and 2023.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of four.

2023

PaLM: Scaling Language Modeling with Pathways.

Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengcheng Yin, Toju Duke, Anselm Levskaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier Garcia, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito, David Luan, Hyeontaek Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, Andrew M. Dai, Thanumalayan Sankaranarayana Pillai, Marie Pellat, Aitor Lewkowycz, Erica Moreira, Rewon Child, Oleksandr Polozov, Katherine Lee, Zongwei Zhou, Xuezhi Wang, Brennan Saeta, Mark Diaz, Orhan Firat, Michele Catasta, Jason Wei, Kathy Meier-Hellstern, Douglas Eck, Jeff Dean, Slav Petrov, Noah Fiedel

J. Mach. Learn. Res., 2023

2022

Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity.

William Fedus, Barret Zoph, Noam Shazeer

J. Mach. Learn. Res., 2022

Scaling Up Models and Data with t5x and seqio.

Adam Roberts, Hyung Won Chung, Anselm Levskaya, Gaurav Mishra, James Bradbury, Daniel Andor, Sharan Narang, Brian Lester, Colin Gaffney, Afroz Mohiuddin, Curtis Hawthorne, Aitor Lewkowycz, Alex Salcianu, Marc van Zee, Jacob Austin, Sebastian Goodman, Livio Baldini Soares, Haitang Hu, Sasha Tsvyashchenko, Aakanksha Chowdhery, Jasmijn Bastings, Jannis Bulian, Xavier Garcia, Jianmo Ni, Andrew Chen, Kathleen Kenealy, Jonathan H. Clark, Stephan Lee, Dan Garrette, James Lee-Thorp, Colin Raffel, Noam Shazeer, Marvin Ritter, Maarten Bosma, Alexandre Passos, Jeremy Maitin-Shepard, Noah Fiedel, Mark Omernick, Brennan Saeta, Ryan Sepassi, Alexander Spiridonov, Joshua Newlan, Andrea Gesmundo

CoRR, 2022

Designing Effective Sparse Expert Models.

Barret Zoph, Irwan Bello, Sameer Kumar, Nan Du, Yanping Huang, Jeff Dean, Noam Shazeer, William Fedus

CoRR, 2022

LaMDA: Language Models for Dialog Applications.

Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, YaGuang Li, Hongrae Lee, Huaixiu Steven Zheng, Amin Ghafouri, Marcelo Menegali, Yanping Huang, Maxim Krikun, Dmitry Lepikhin, James Qin, Dehao Chen, Yuanzhong Xu, Zhifeng Chen, Adam Roberts, Maarten Bosma, Yanqi Zhou, Chung-Ching Chang, Igor Krivokon, Will Rusch, Marc Pickett, Kathleen S. Meier-Hellstern, Meredith Ringel Morris, Tulsee Doshi, Renelito Delos Santos, Toju Duke, Johnny Soraker, Ben Zevenbergen, Vinodkumar Prabhakaran, Mark Diaz, Ben Hutchinson, Kristen Olson, Alejandra Molina, Erin Hoffman-John, Josh Lee, Lora Aroyo, Ravi Rajakumar, Alena Butryna, Matthew Lamm, Viktoriya Kuzmina, Joe Fenton, Aaron Cohen, Rachel Bernstein, Ray Kurzweil, Blaise Agüera y Arcas, Claire Cui, Marian Croak, Ed H. Chi, Quoc Le

CoRR, 2022

2021

Primer: Searching for Efficient Transformers for Language Modeling.

David R. So, Wojciech Manke, Hanxiao Liu, Zihang Dai, Noam Shazeer, Quoc V. Le

CoRR, 2021

GSPMD: General and Scalable Parallelization for ML Computation Graphs.

Yuanzhong Xu, HyoukJoong Lee, Dehao Chen, Blake A. Hechtman, Yanping Huang, Rahul Joshi, Maxim Krikun, Dmitry Lepikhin, Andy Ly, Marcello Maggioni, Ruoming Pang, Noam Shazeer, Shibo Wang, Tao Wang, Yonghui Wu, Zhifeng Chen

CoRR, 2021

Searching for Efficient Transformers for Language Modeling.

David R. So, Wojciech Manke, Hanxiao Liu, Zihang Dai, Noam Shazeer, Quoc V. Le

Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, 2021

GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding.

Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen, Orhan Firat, Yanping Huang, Maxim Krikun, Noam Shazeer, Zhifeng Chen

Proceedings of the 9th International Conference on Learning Representations, 2021

Do Transformer Modifications Transfer Across Implementations and Applications?

Sharan Narang, Hyung Won Chung, Yi Tay, Liam Fedus, Thibault Févry, Michael Matena, Karishma Malkan, Noah Fiedel, Noam Shazeer, Zhenzhong Lan, Yanqi Zhou, Wei Li, Nan Ding, Jake Marcus, Adam Roberts, Colin Raffel

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021

2020

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer.

Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu

J. Mach. Learn. Res., 2020

Talking-Heads Attention.

Noam Shazeer, Zhenzhong Lan, Youlong Cheng, Nan Ding, Le Hou

CoRR, 2020

GLU Variants Improve Transformer.

Noam Shazeer

CoRR, 2020

Faster Transformer Decoding: N-gram Masked Self-Attention.

Ciprian Chelba, Mia Xu Chen, Ankur Bapna, Noam Shazeer

CoRR, 2020

How Much Knowledge Can You Pack Into the Parameters of a Language Model?

Adam Roberts, Colin Raffel, Noam Shazeer

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, 2020

2019

Fast Transformer Decoding: One Write-Head is All You Need.

Noam Shazeer

CoRR, 2019

High Resolution Medical Image Analysis with Spatial Partitioning.

Le Hou, Youlong Cheng, Noam Shazeer, Niki Parmar, Yeqing Li, Panagiotis Korfiatis, Travis M. Drucker, Daniel J. Blezek, Xiaodan Song

CoRR, 2019

Corpora Generation for Grammatical Error Correction.

Jared Lichtarge, Chris Alberti, Shankar Kumar, Noam Shazeer, Niki Parmar, Simon Tong

Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019

Music Transformer: Generating Music with Long-Term Structure.

Cheng-Zhi Anna Huang, Ashish Vaswani, Jakob Uszkoreit, Ian Simon, Curtis Hawthorne, Noam Shazeer, Andrew M. Dai, Matthew D. Hoffman, Monica Dinculescu, Douglas Eck

Proceedings of the 7th International Conference on Learning Representations, 2019

2018

Weakly Supervised Grammatical Error Correction using Iterative Decoding.

Jared Lichtarge, Christopher Alberti, Shankar Kumar, Noam Shazeer, Niki Parmar

CoRR, 2018

An Improved Relative Self-Attention Mechanism for Transformer with Application to Music Generation.

Cheng-Zhi Anna Huang, Ashish Vaswani, Jakob Uszkoreit, Noam Shazeer, Curtis Hawthorne, Andrew M. Dai, Matthew D. Hoffman, Douglas Eck

CoRR, 2018

Image Transformer.

Niki Parmar, Ashish Vaswani, Jakob Uszkoreit, Lukasz Kaiser, Noam Shazeer, Alexander Ku

CoRR, 2018

Blockwise Parallel Decoding for Deep Autoregressive Models.

Mitchell Stern, Noam Shazeer, Jakob Uszkoreit

Proceedings of the Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, 2018

Mesh-TensorFlow: Deep Learning for Supercomputers.

Noam Shazeer, Youlong Cheng, Niki Parmar, Dustin Tran, Ashish Vaswani, Penporn Koanantakool, Peter Hawkins, HyoukJoong Lee, Mingsheng Hong, Cliff Young, Ryan Sepassi, Blake A. Hechtman

Proceedings of the Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, 2018

Adafactor: Adaptive Learning Rates with Sublinear Memory Cost.

Noam Shazeer, Mitchell Stern

Proceedings of the 35th International Conference on Machine Learning, 2018

Image Transformer.

Niki Parmar, Ashish Vaswani, Jakob Uszkoreit, Lukasz Kaiser, Noam Shazeer, Alexander Ku, Dustin Tran

Proceedings of the 35th International Conference on Machine Learning, 2018

Fast Decoding in Sequence Models Using Discrete Latent Variables.

Lukasz Kaiser, Samy Bengio, Aurko Roy, Ashish Vaswani, Niki Parmar, Jakob Uszkoreit, Noam Shazeer

Proceedings of the 35th International Conference on Machine Learning, 2018

Generating Wikipedia by Summarizing Long Sequences.

Peter J. Liu, Mohammad Saleh, Etienne Pot, Ben Goodrich, Ryan Sepassi, Lukasz Kaiser, Noam Shazeer

Proceedings of the 6th International Conference on Learning Representations, 2018

HydraNets: Specialized Dynamic Architectures for Efficient Inference.

Ravi Teja Mullapudi, William R. Mark, Noam Shazeer, Kayvon Fatahalian

Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, 2018

Tensor2Tensor for Neural Machine Translation.

Ashish Vaswani, Samy Bengio, Eugene Brevdo, François Chollet, Aidan N. Gomez, Stephan Gouws, Llion Jones, Lukasz Kaiser, Nal Kalchbrenner, Niki Parmar, Ryan Sepassi, Noam Shazeer, Jakob Uszkoreit

Proceedings of the 13th Conference of the Association for Machine Translation in the Americas, 2018

The Best of Both Worlds: Combining Recent Advances in Neural Machine Translation.

Mia Xu Chen, Orhan Firat, Ankur Bapna, Melvin Johnson, Wolfgang Macherey, George F. Foster, Llion Jones, Mike Schuster, Noam Shazeer, Niki Parmar, Ashish Vaswani, Jakob Uszkoreit, Lukasz Kaiser, Zhifeng Chen, Yonghui Wu, Macduff Hughes

Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2018

2017

One Model To Learn Them All.

Lukasz Kaiser, Aidan N. Gomez, Noam Shazeer, Ashish Vaswani, Niki Parmar, Llion Jones, Jakob Uszkoreit

CoRR, 2017

Attention is All you Need.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin

Proceedings of the Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 2017

Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer.

Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc V. Le, Geoffrey E. Hinton, Jeff Dean

Proceedings of the 5th International Conference on Learning Representations, 2017

2016

Sparse Non-negative Matrix Language Modeling.

Joris Pelemans, Noam Shazeer, Ciprian Chelba

Trans. Assoc. Comput. Linguistics, 2016

Swivel: Improving Embeddings by Noticing What's Missing.

Noam Shazeer, Ryan Doherty, Colin Evans, Chris Waterson

CoRR, 2016

Exploring the Limits of Language Modeling.

Rafal Józefowicz, Oriol Vinyals, Mike Schuster, Noam Shazeer, Yonghui Wu

CoRR, 2016

NN-Grams: Unifying Neural Network and n-Gram Language Models for Speech Recognition.

Babak Damavandi, Shankar Kumar, Noam Shazeer, Antoine Bruguier

Proceedings of the Interspeech 2016, 2016

End-to-end text-dependent speaker verification.

Georg Heigold, Ignacio Moreno, Samy Bengio, Noam Shazeer

Proceedings of the 2016 IEEE International Conference on Acoustics, 2016

2015

Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks.

Samy Bengio, Oriol Vinyals, Navdeep Jaitly, Noam Shazeer

Proceedings of the Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, 2015

Sparse non-negative matrix language modeling for skip-grams.

Noam Shazeer, Joris Pelemans, Ciprian Chelba

Proceedings of the INTERSPEECH 2015, 2015

Pruning sparse non-negative matrix n-gram language models.

Joris Pelemans, Noam Shazeer, Ciprian Chelba

Proceedings of the INTERSPEECH 2015, 2015

Sparse non-negative matrix language modeling for geo-annotated query session data.

Ciprian Chelba, Noam Shazeer

Proceedings of the 2015 IEEE Workshop on Automatic Speech Recognition and Understanding, 2015

2014

Skip-gram Language Modeling Using Sparse Non-negative Matrix Probability Estimation.

Noam Shazeer, Joris Pelemans, Ciprian Chelba

CoRR, 2014

2010

Variational Program Inference

Georges Harik, Noam Shazeer

CoRR, 2010

2002

A probabilistic approach to solving crossword puzzles.

Michael L. Littman, Greg A. Keim, Noam M. Shazeer

Artif. Intell., 2002

1999

Solving Crossword Puzzles as Probabilistic Constraint Satisfaction.

Noam M. Shazeer, Michael L. Littman, Greg A. Keim

Proceedings of the Sixteenth National Conference on Artificial Intelligence and Eleventh Conference on Innovative Applications of Artificial Intelligence, 1999

Solving Crosswords with PROVERB.

Michael L. Littman, Greg A. Keim, Noam M. Shazeer

Proceedings of the Sixteenth National Conference on Artificial Intelligence and Eleventh Conference on Innovative Applications of Artificial Intelligence, 1999

PROVERB: The Probabilistic Cruciverbalist.

Greg A. Keim, Noam M. Shazeer, Michael L. Littman, Sushant Agarwal, Catherine M. Cheves, Joseph Fitzgerald, Jason Grosland, Fan Jiang, Shannon Pollard, Karl Weinmeister

Proceedings of the Sixteenth National Conference on Artificial Intelligence and Eleventh Conference on Innovative Applications of Artificial Intelligence, 1999