Blake A. Hechtman

According to our database, Blake A. Hechtman authored at least 17 papers between 2013 and 2022.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of four.

2022

Unified Scaling Laws for Routed Language Models.

Aidan Clark, Diego de Las Casas, Aurelia Guy, Arthur Mensch, Michela Paganini, Jordan Hoffmann, Bogdan Damoc, Blake A. Hechtman, Trevor Cai, Sebastian Borgeaud, George van den Driessche, Eliza Rutherford, Tom Hennigan, Matthew J. Johnson, Katie Millican, Albin Cassirer, Chris Jones, Elena Buchatskaya, David Budden, Laurent Sifre, Simon Osindero, Oriol Vinyals, Jack W. Rae, Erich Elsen, Koray Kavukcuoglu, Karen Simonyan

CoRR, 2022

Unified Scaling Laws for Routed Language Models.

Aidan Clark, Diego de Las Casas, Aurelia Guy, Arthur Mensch, Michela Paganini, Jordan Hoffmann, Bogdan Damoc, Blake A. Hechtman, Trevor Cai, Sebastian Borgeaud, George van den Driessche, Eliza Rutherford, Tom Hennigan, Matthew J. Johnson, Albin Cassirer, Chris Jones, Elena Buchatskaya, David Budden, Laurent Sifre, Simon Osindero, Oriol Vinyals, Marc'Aurelio Ranzato, Jack W. Rae, Erich Elsen, Koray Kavukcuoglu, Karen Simonyan

Proceedings of the International Conference on Machine Learning, 2022

2021

Scaling Language Models: Methods, Analysis & Insights from Training Gopher.

Jack W. Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, H. Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, Eliza Rutherford, Tom Hennigan, Jacob Menick, Albin Cassirer, Richard Powell, George van den Driessche, Lisa Anne Hendricks, Maribeth Rauh, Po-Sen Huang, Amelia Glaese, Johannes Welbl, Sumanth Dathathri, Saffron Huang, Jonathan Uesato, John Mellor, Irina Higgins, Antonia Creswell, Nat McAleese, Amy Wu, Erich Elsen, Siddhant M. Jayakumar, Elena Buchatskaya, David Budden, Esme Sutherland, Karen Simonyan, Michela Paganini, Laurent Sifre, Lena Martens, Xiang Lorraine Li, Adhiguna Kuncoro, Aida Nematzadeh, Elena Gribovskaya, Domenic Donato, Angeliki Lazaridou, Arthur Mensch, Jean-Baptiste Lespiau, Maria Tsimpoukelli, Nikolai Grigorev, Doug Fritz, Thibault Sottiaux, Mantas Pajarskas, Toby Pohlen, Zhitao Gong, Daniel Toyama, Cyprien de Masson d'Autume, Yujia Li, Tayfun Terzi, Vladimir Mikulik, Igor Babuschkin, Aidan Clark, Diego de Las Casas, Aurelia Guy, Chris Jones, James Bradbury, Matthew J. Johnson, Blake A. Hechtman, Laura Weidinger, Iason Gabriel, William Isaac, Edward Lockhart, Simon Osindero, Laura Rimell, Chris Dyer, Oriol Vinyals, Kareem Ayoub, Jeff Stanway, Lorrayne Bennett, Demis Hassabis, Koray Kavukcuoglu, Geoffrey Irving

CoRR, 2021

GSPMD: General and Scalable Parallelization for ML Computation Graphs.

Yuanzhong Xu, HyoukJoong Lee, Dehao Chen, Blake A. Hechtman, Yanping Huang, Rahul Joshi, Maxim Krikun, Dmitry Lepikhin, Andy Ly, Marcello Maggioni, Ruoming Pang, Noam Shazeer, Shibo Wang, Tao Wang, Yonghui Wu, Zhifeng Chen

CoRR, 2021

Large-Scale Discrete Fourier Transform on TPUs.

Tianjian Lu, Yi-Fan Chen, Blake A. Hechtman, Tao Wang, John R. Anderson

IEEE Access, 2021

Scaling Local Self-Attention for Parameter Efficient Visual Backbones.

Ashish Vaswani, Prajit Ramachandran, Aravind Srinivas, Niki Parmar, Blake A. Hechtman, Jonathon Shlens

Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2021

A Flexible Approach to Autotuning Multi-Pass Machine Learning Compilers.

Phitchaya Mangpo Phothilimthana, Amit Sabne, Nikhil Sarda, Karthik Srinivasa Murthy, Yanqi Zhou, Christof Angermueller, Mike Burrows, Sudip Roy, Ketan Mandke, Rezsa Farahani, Yu Emma Wang, Berkin Ilbeyi, Blake A. Hechtman, Bjarke Roune, Shen Wang, Yuanzhong Xu, Samuel J. Kaufman

Proceedings of the 30th International Conference on Parallel Architectures and Compilation Techniques, 2021

2020

Exploring the limits of Concurrency in ML Training on Google TPUs.

Sameer Kumar, James Bradbury, Cliff Young, Yu Emma Wang, Anselm Levskaya, Blake A. Hechtman, Dehao Chen, HyoukJoong Lee, Mehmet Deveci, Naveen Kumar, Pankaj Kanwar, Shibo Wang, Skye Wanderman-Milne, Steve Lacy, Tao Wang, Tayo Oguntebi, Yazhou Zu, Yuanzhong Xu, Andy Swing

CoRR, 2020

Automatic Cross-Replica Sharding of Weight Update in Data-Parallel Training.

Yuanzhong Xu, HyoukJoong Lee, Dehao Chen, Hongjun Choi, Blake A. Hechtman, Shibo Wang

CoRR, 2020

2019

Scale MLPerf-0.6 models on Google TPU-v3 Pods.

Sameer Kumar, Victor Bitorff, Dehao Chen, Chiachen Chou, Blake A. Hechtman, HyoukJoong Lee, Naveen Kumar, Peter Mattson, Shibo Wang, Tao Wang, Yuanzhong Xu, Zongwei Zhou

CoRR, 2019

2018

Mesh-TensorFlow: Deep Learning for Supercomputers.

Noam Shazeer, Youlong Cheng, Niki Parmar, Dustin Tran, Ashish Vaswani, Penporn Koanantakool, Peter Hawkins, HyoukJoong Lee, Mingsheng Hong, Cliff Young, Ryan Sepassi, Blake A. Hechtman

Proceedings of the Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, 2018

2016

TREES: A CPU/GPU Task-Parallel Runtime with Explicit Epoch Synchronization.

Blake A. Hechtman, Andrew D. Hilton, Daniel J. Sorin

CoRR, 2016

2014

Exploiting Parallelism in GPUs.

Blake A. Hechtman

PhD thesis, 2014

QuickRelease: A throughput-oriented approach to release consistency on GPUs.

Blake A. Hechtman, Shuai Che, Derek R. Hower, Yingying Tian, Bradford M. Beckmann, Mark D. Hill, Steven K. Reinhardt, David A. Wood

Proceedings of the 20th IEEE International Symposium on High Performance Computer Architecture, 2014

Heterogeneous-race-free memory models.

Derek R. Hower, Blake A. Hechtman, Bradford M. Beckmann, Benedict R. Gaster, Mark D. Hill, Steven K. Reinhardt, David A. Wood

Proceedings of the Architectural Support for Programming Languages and Operating Systems, 2014

2013

Evaluating cache coherent shared virtual memory for heterogeneous multicore chips.

Blake A. Hechtman, Daniel J. Sorin

Proceedings of the 2012 IEEE International Symposium on Performance Analysis of Systems & Software, 2013

Exploring memory consistency for massively-threaded throughput-oriented processors.

Blake A. Hechtman, Daniel J. Sorin

Proceedings of the 40th Annual International Symposium on Computer Architecture, 2013