Erich Elsen

According to our database, Erich Elsen authored at least 37 papers between 2006 and 2022.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of four.

2022

Training Compute-Optimal Large Language Models.

Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, Erich Elsen, Jack W. Rae, Oriol Vinyals, Laurent Sifre

CoRR, 2022

Unified Scaling Laws for Routed Language Models.

Aidan Clark, Diego de Las Casas, Aurelia Guy, Arthur Mensch, Michela Paganini, Jordan Hoffmann, Bogdan Damoc, Blake A. Hechtman, Trevor Cai, Sebastian Borgeaud, George van den Driessche, Eliza Rutherford, Tom Hennigan, Matthew J. Johnson, Katie Millican, Albin Cassirer, Chris Jones, Elena Buchatskaya, David Budden, Laurent Sifre, Simon Osindero, Oriol Vinyals, Jack W. Rae, Erich Elsen, Koray Kavukcuoglu, Karen Simonyan

CoRR, 2022

An empirical analysis of compute-optimal large language model training.

Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katherine Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, Erich Elsen, Oriol Vinyals, Jack W. Rae, Laurent Sifre

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

The State of Sparse Training in Deep Reinforcement Learning.

Laura Graesser, Utku Evci, Erich Elsen, Pablo Samuel Castro

Proceedings of the International Conference on Machine Learning, 2022

Unified Scaling Laws for Routed Language Models.

Aidan Clark, Diego de Las Casas, Aurelia Guy, Arthur Mensch, Michela Paganini, Jordan Hoffmann, Bogdan Damoc, Blake A. Hechtman, Trevor Cai, Sebastian Borgeaud, George van den Driessche, Eliza Rutherford, Tom Hennigan, Matthew J. Johnson, Albin Cassirer, Chris Jones, Elena Buchatskaya, David Budden, Laurent Sifre, Simon Osindero, Oriol Vinyals, Marc'Aurelio Ranzato, Jack W. Rae, Erich Elsen, Koray Kavukcuoglu, Karen Simonyan

Proceedings of the International Conference on Machine Learning, 2022

Improving Language Models by Retrieving from Trillions of Tokens.

Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George van den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, Diego de Las Casas, Aurelia Guy, Jacob Menick, Roman Ring, Tom Hennigan, Saffron Huang, Loren Maggiore, Chris Jones, Albin Cassirer, Andy Brock, Michela Paganini, Geoffrey Irving, Oriol Vinyals, Simon Osindero, Karen Simonyan, Jack W. Rae, Erich Elsen, Laurent Sifre

Proceedings of the International Conference on Machine Learning, 2022

Step-unrolled Denoising Autoencoders for Text Generation.

Nikolay Savinov, Junyoung Chung, Mikolaj Binkowski, Erich Elsen, Aäron van den Oord

Proceedings of the Tenth International Conference on Learning Representations, 2022

2021

Scaling Language Models: Methods, Analysis & Insights from Training Gopher.

Jack W. Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, H. Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, Eliza Rutherford, Tom Hennigan, Jacob Menick, Albin Cassirer, Richard Powell, George van den Driessche, Lisa Anne Hendricks, Maribeth Rauh, Po-Sen Huang, Amelia Glaese, Johannes Welbl, Sumanth Dathathri, Saffron Huang, Jonathan Uesato, John Mellor, Irina Higgins, Antonia Creswell, Nat McAleese, Amy Wu, Erich Elsen, Siddhant M. Jayakumar, Elena Buchatskaya, David Budden, Esme Sutherland, Karen Simonyan, Michela Paganini, Laurent Sifre, Lena Martens, Xiang Lorraine Li, Adhiguna Kuncoro, Aida Nematzadeh, Elena Gribovskaya, Domenic Donato, Angeliki Lazaridou, Arthur Mensch, Jean-Baptiste Lespiau, Maria Tsimpoukelli, Nikolai Grigorev, Doug Fritz, Thibault Sottiaux, Mantas Pajarskas, Toby Pohlen, Zhitao Gong, Daniel Toyama, Cyprien de Masson d'Autume, Yujia Li, Tayfun Terzi, Vladimir Mikulik, Igor Babuschkin, Aidan Clark, Diego de Las Casas, Aurelia Guy, Chris Jones, James Bradbury, Matthew J. Johnson, Blake A. Hechtman, Laura Weidinger, Iason Gabriel, William Isaac, Edward Lockhart, Simon Osindero, Laura Rimell, Chris Dyer, Oriol Vinyals, Kareem Ayoub, Jeff Stanway, Lorrayne Bennett, Demis Hassabis, Koray Kavukcuoglu, Geoffrey Irving

CoRR, 2021

Practical Real Time Recurrent Learning with a Sparse Approximation.

Jacob Menick, Erich Elsen, Utku Evci, Simon Osindero, Karen Simonyan, Alex Graves

Proceedings of the 9th International Conference on Learning Representations, 2021

End-to-end Adversarial Text-to-Speech.

Jeff Donahue, Sander Dieleman, Mikolaj Binkowski, Erich Elsen, Karen Simonyan

Proceedings of the 9th International Conference on Learning Representations, 2021

2020

AlgebraNets.

Jordan Hoffmann, Simon Schmitt, Simon Osindero, Karen Simonyan, Erich Elsen

CoRR, 2020

A Practical Sparse Approximation for Real Time Recurrent Learning.

Jacob Menick, Erich Elsen, Utku Evci, Simon Osindero, Karen Simonyan, Alex Graves

CoRR, 2020

Sparse GPU kernels for deep learning.

Trevor Gale, Matei Zaharia, Cliff Young, Erich Elsen

Proceedings of the International Conference for High Performance Computing, 2020

Top-KAST: Top-K Always Sparse Training.

Siddhant M. Jayakumar, Razvan Pascanu, Jack W. Rae, Simon Osindero, Erich Elsen

Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, 2020

On the Generalization Benefit of Noise in Stochastic Gradient Descent.

Samuel L. Smith, Erich Elsen, Soham De

Proceedings of the 37th International Conference on Machine Learning, 2020

Rigging the Lottery: Making All Tickets Winners.

Utku Evci, Trevor Gale, Jacob Menick, Pablo Samuel Castro, Erich Elsen

Proceedings of the 37th International Conference on Machine Learning, 2020

High Fidelity Speech Synthesis with Adversarial Networks.

Mikolaj Binkowski, Jeff Donahue, Sander Dieleman, Aidan Clark, Erich Elsen, Norman Casagrande, Luis C. Cobo, Karen Simonyan

Proceedings of the 8th International Conference on Learning Representations, 2020

Fast Sparse ConvNets.

Erich Elsen, Marat Dukhan, Trevor Gale, Karen Simonyan

Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020

2019

The Difficulty of Training Sparse Neural Networks.

Utku Evci, Fabian Pedregosa, Aidan N. Gomez, Erich Elsen

CoRR, 2019

Non-Differentiable Supervised Learning with Evolution Strategies and Hybrid Methods.

Karel Lenc, Erich Elsen, Tom Schaul, Karen Simonyan

CoRR, 2019

The State of Sparsity in Deep Neural Networks.

Trevor Gale, Erich Elsen, Sara Hooker

CoRR, 2019

Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Dataset.

Curtis Hawthorne, Andriy Stasyuk, Adam Roberts, Ian Simon, Cheng-Zhi Anna Huang, Sander Dieleman, Erich Elsen, Jesse H. Engel, Douglas Eck

Proceedings of the 7th International Conference on Learning Representations, 2019

2018

Onsets and Frames: Dual-Objective Piano Transcription.

Curtis Hawthorne, Erich Elsen, Jialin Song, Adam Roberts, Ian Simon, Colin Raffel, Jesse H. Engel, Sageev Oore, Douglas Eck

Proceedings of the 19th International Society for Music Information Retrieval Conference, 2018

Parallel WaveNet: Fast High-Fidelity Speech Synthesis.

Aäron van den Oord, Yazhe Li, Igor Babuschkin, Karen Simonyan, Oriol Vinyals, Koray Kavukcuoglu, George van den Driessche, Edward Lockhart, Luis C. Cobo, Florian Stimberg, Norman Casagrande, Dominik Grewe, Seb Noury, Sander Dieleman, Erich Elsen, Nal Kalchbrenner, Heiga Zen, Alex Graves, Helen King, Tom Walters, Dan Belov, Demis Hassabis

Proceedings of the 35th International Conference on Machine Learning, 2018

Efficient Neural Audio Synthesis.

Nal Kalchbrenner, Erich Elsen, Karen Simonyan, Seb Noury, Norman Casagrande, Edward Lockhart, Florian Stimberg, Aäron van den Oord, Sander Dieleman, Koray Kavukcuoglu

Proceedings of the 35th International Conference on Machine Learning, 2018

Mixed Precision Training.

Paulius Micikevicius, Sharan Narang, Jonah Alben, Gregory F. Diamos, Erich Elsen, David García, Boris Ginsburg, Michael Houston, Oleksii Kuchaiev, Ganesh Venkatesh, Hao Wu

Proceedings of the 6th International Conference on Learning Representations, 2018

2017

Exploring Sparsity in Recurrent Neural Networks.

Sharan Narang, Greg Diamos, Shubho Sengupta, Erich Elsen

Proceedings of the 5th International Conference on Learning Representations, 2017

DSD: Dense-Sparse-Dense Training for Deep Neural Networks.

Song Han, Jeff Pool, Sharan Narang, Huizi Mao, Enhao Gong, Shijian Tang, Erich Elsen, Peter Vajda, Manohar Paluri, John Tran, Bryan Catanzaro, William J. Dally

Proceedings of the 5th International Conference on Learning Representations, 2017

2016

DSD: Regularizing Deep Neural Networks with Dense-Sparse-Dense Training Flow.

Song Han, Jeff Pool, Sharan Narang, Huizi Mao, Shijian Tang, Erich Elsen, Bryan Catanzaro, John Tran, William J. Dally

CoRR, 2016

Persistent RNNs: Stashing Recurrent Weights On-Chip.

Greg Diamos, Shubho Sengupta, Bryan Catanzaro, Mike Chrzanowski, Adam Coates, Erich Elsen, Jesse H. Engel, Awni Y. Hannun, Sanjeev Satheesh

Proceedings of the 33nd International Conference on Machine Learning, 2016

Deep Speech 2 : End-to-End Speech Recognition in English and Mandarin.

Dario Amodei, Sundaram Ananthanarayanan, Rishita Anubhai, Jingliang Bai, Eric Battenberg, Carl Case, Jared Casper, Bryan Catanzaro, Jingdong Chen, Mike Chrzanowski, Adam Coates, Greg Diamos, Erich Elsen, Jesse H. Engel, Linxi Fan, Christopher Fougner, Awni Y. Hannun, Billy Jun, Tony Han, Patrick LeGresley, Xiangang Li, Libby Lin, Sharan Narang, Andrew Y. Ng, Sherjil Ozair, Ryan Prenger, Sheng Qian, Jonathan Raiman, Sanjeev Satheesh, David Seetapun, Shubho Sengupta, Chong Wang, Yi Wang, Zhiqian Wang, Bo Xiao, Yan Xie, Dani Yogatama, Jun Zhan, Zhenyao Zhu

Proceedings of the 33nd International Conference on Machine Learning, 2016

2015

Deep Speech 2: End-to-End Speech Recognition in English and Mandarin.

Dario Amodei, Rishita Anubhai, Eric Battenberg, Carl Case, Jared Casper, Bryan Catanzaro, Jingdong Chen, Mike Chrzanowski, Adam Coates, Greg Diamos, Erich Elsen, Jesse H. Engel, Linxi Fan, Christopher Fougner, Tony Han, Awni Y. Hannun, Billy Jun, Patrick LeGresley, Libby Lin, Sharan Narang, Andrew Y. Ng, Sherjil Ozair, Ryan Prenger, Jonathan Raiman, Sanjeev Satheesh, David Seetapun, Shubho Sengupta, Yi Wang, Zhiqian Wang, Chong Wang, Bo Xiao, Dani Yogatama, Jun Zhan, Zhenyao Zhu

CoRR, 2015

2014

Deep Speech: Scaling up end-to-end speech recognition.

Awni Y. Hannun, Carl Case, Jared Casper, Bryan Catanzaro, Greg Diamos, Erich Elsen, Ryan Prenger, Sanjeev Satheesh, Shubho Sengupta, Adam Coates, Andrew Y. Ng

CoRR, 2014

2011

Liszt: a domain specific language for building portable mesh-based PDE solvers.

Zach DeVito, Niels Joubert, Francisco Palacios, Stephen Oakley, Montserrat Medina, Mike Barrientos, Erich Elsen, Frank Ham, Alex Aiken, Karthik Duraisamy, Eric Darve, Juan J. Alonso, Pat Hanrahan

Proceedings of the Conference on High Performance Computing Networking, 2011

2008

Large calculation of the flow over a hypersonic vehicle using a GPU.

Erich Elsen, Patrick LeGresley, Eric Darve

J. Comput. Phys., 2008

2007

N-Body Simulations on GPUs

Erich Elsen, Vaidyanathan Vishal, Mike Houston, Vijay S. Pande, Pat Hanrahan, Eric Darve

CoRR, 2007

2006

Poster reception - N-Body simulation on GPUs.

Erich Elsen, Mike Houston, Vaidyanathan Vishal, Eric Darve, Pat Hanrahan, Vijay S. Pande

Proceedings of the ACM/IEEE SC2006 Conference on High Performance Networking and Computing, 2006