Express Cube Topologies for on-Chip Interconnects. - Boris Grot; Joel Hestness; Stephen W. Keckler; Onur Mutlu (HPCA, 2009)

Netrace: dependency-driven trace-based network-on-chip simulation. - Joel Hestness; Boris Grot; Stephen W. Keckler (NoCArc@MICRO, 2010)

Kilo-NOC: a heterogeneous network-on-chip architecture for scalability and service guarantees. - Boris Grot; Joel Hestness; Stephen W. Keckler; Onur Mutlu (ISCA, 2011)

The gem5 simulator. - Nathan L. Binkert; Bradford M. Beckmann; Gabriel Black; Steven K. Reinhardt; Ali G. Saidi; Arkaprava Basu; Joel Hestness; Derek Hower; Tushar Krishna; Somayeh Sardashti; Rathijit Sen; Korey Sewell; et al. (SIGARCH Comput. Archit. News, 2011)

A QoS-Enabled On-Die Interconnect Fabric for Kilo-Node Chips. - Boris Grot; Joel Hestness; Stephen W. Keckler; Onur Mutlu (IEEE Micro, 2012)

A comparative analysis of microarchitecture effects on CPU and GPU memory system behavior. - Joel Hestness; Stephen W. Keckler; David A. Wood 001 (IISWC, 2014)

GPU Computing Pipeline Inefficiencies and Optimization Opportunities in Heterogeneous CPU-GPU Processors. - Joel Hestness; Stephen W. Keckler; David A. Wood 001 (IISWC, 2015)

gem5-gpu: A Heterogeneous CPU-GPU Simulator. - Jason Power; Joel Hestness; Marc S. Orr; Mark D. Hill; David A. Wood 001 (IEEE Comput. Archit. Lett., 2015)

Convolutional Recurrent Neural Networks for Small-Footprint Keyword Spotting. - Sercan Ömer Arik; Markus Kliegl; Rewon Child; Joel Hestness; Andrew Gibiansky; Christopher Fougner; Ryan Prenger; Adam Coates 002 (INTERSPEECH, 2017)

Deep Learning Scaling is Predictable, Empirically. - Joel Hestness; Sharan Narang; Newsha Ardalani; Gregory F. Diamos; Heewoo Jun; Hassan Kianinejad; Md. Mostofa Ali Patwary; Yang Yang; Yanqi Zhou (CoRR, 2017)

Compositional Generalization for Primitive Substitutions. - Yuanpeng Li 001; Liang Zhao; Jianyu Wang; Joel Hestness (EMNLP/IJCNLP (1), 2019)

Beyond human-level accuracy: computational challenges in deep learning. - Joel Hestness; Newsha Ardalani; Gregory F. Diamos (PPoPP, 2019)

A survey of 25 years of evaluation. - Kenneth Ward Church; Joel Hestness (Nat. Lang. Eng., 2019)

Memory Efficient 3D U-Net with Reversible Mobile Inverted Bottlenecks for Brain Tumor Segmentation. - Mihir Pendse; Vithursan Thangarasa; Vitaliy Chiley; Ryan Holmdahl; Joel Hestness; Dennis DeCoste (BrainLes@MICCAI (2), 2020)

Pipelined Backpropagation at Scale: Training Large Models without Batches. - Atli Kosson; Vitaliy Chiley; Abhinav Venigalla; Joel Hestness; Urs Köster (MLSys, 2021)

Time and the Value of Data. - Ehsan Valavi; Joel Hestness; Newsha Ardalani; Marco Iansiti (CoRR, 2022)

Time Dependency, Data Flow, and Competitive Advantage. - Ehsan Valavi; Joel Hestness; Marco Iansiti; Newsha Ardalani; Feng Zhu 023; Karim R. Lakhani (CoRR, 2022)

RevBiFPN: The Fully Reversible Bidirectional Feature Pyramid Network. - Vitaliy Chiley; Vithursan Thangarasa; Abhay Gupta; Anshul Samar; Joel Hestness; Dennis DeCoste (MLSys, 2023)

Cerebras-GPT: Open Compute-Optimal Language Models Trained on the Cerebras Wafer-Scale Cluster. - Nolan Dey; Gurpreet Gosal; Zhiming Chen; Hemant Khachane; William Marshall; Ribhu Pathria; Marvin Tom; Joel Hestness (CoRR, 2023)

SlimPajama-DC: Understanding Data Combinations for LLM Training. - Zhiqiang Shen; Tianhua Tao; Liqun Ma; Willie Neiswanger; Zhengzhong Liu 001; Hongyi Wang 001; Bowen Tan; Joel Hestness; Natalia Vassilieva; Daria Soboleva; Eric P. Xing (CoRR, 2023)

BTLM-3B-8K: 7B Parameter Performance in a 3B Parameter Model. - Nolan Dey; Daria Soboleva; Faisal Al-Khateeb; Bowen Yang; Ribhu Pathria; Hemant Khachane; Shaheer Muhammad; Zhiming Chen; Robert Myers; Jacob Robert Steeves; Natalia Vassilieva; Marvin Tom; et al. (CoRR, 2023)

Position Interpolation Improves ALiBi Extrapolation. - Faisal Al-Khateeb; Nolan Dey; Daria Soboleva; Joel Hestness (CoRR, 2023)

MediSwift: Efficient Sparse Pre-trained Biomedical Language Models. - Vithursan Thangarasa; Mahmoud Salem; Shreyas Saxena; Chen-Yu Leong; Joel Hestness; Sean Lie (ACL (Findings), 2024)

Efficiently Disentangle Causal Representations. - Yuanpeng Li 001; Joel Hestness; Mohamed Elhoseiny; Liang Zhao; Kenneth Church 001 (CPAL, 2024)

Sparse maximal update parameterization: A holistic approach to sparse training dynamics. - Nolan Dey; Shane Bergsma; Joel Hestness (NeurIPS, 2024)

Normalization Layer Per-Example Gradients are Sufficient to Predict Gradient Noise Scale in Transformers. - Gavia Gray; Aman Tiwari; Shane Bergsma; Joel Hestness (NeurIPS, 2024)

MediSwift: Efficient Sparse Pre-trained Biomedical Language Models. - Vithursan Thangarasa; Mahmoud Salem; Shreyas Saxena; Kevin Leong; Joel Hestness; Sean Lie (CoRR, 2024)

Bilingual Adaptation of Monolingual Foundation Models. - Gurpreet Gosal; Yishi Xu; Gokul Ramakrishnan; Rituraj Joshi; Avraham Sheinin; Zhiming Chen; Biswajit Mishra; Natalia Vassilieva; Joel Hestness; Neha Sengupta; Sunil Kumar Sahu; Bokang Jia; et al. (CoRR, 2024)

Crystal: Illuminating LLM Abilities on Language and Code. - Tianhua Tao; Junbo Li; Bowen Tan; Hongyi Wang 001; William Marshall; Bhargav Kanakiya; Joel Hestness; Natalia Vassilieva; Zhiqiang Shen; Eric P. Xing; Zhengzhong Liu 001 (CoRR, 2024)

Straight to Zero: Why Linearly Decaying the Learning Rate to Zero Works Best for LLMs. - Shane Bergsma; Nolan Simran Dey; Gurpreet Gosal; Gavia Gray; Daria Soboleva; Joel Hestness (ICLR, 2025)

Don't be lazy: CompleteP enables compute-efficient deep transformers. - Nolan Dey; Bin Claire Zhang; Lorenzo Noci; Mufan Bill Li; Blake Bordelon; Shane Bergsma; Cengiz Pehlevan; Boris Hanin; Joel Hestness (CoRR, 2025)

Power Lines: Scaling Laws for Weight Decay and Batch Size in LLM Pre-training. - Shane Bergsma; Nolan Dey; Gurpreet Gosal; Gavia Gray; Daria Soboleva; Joel Hestness (CoRR, 2025)