Nolan Dey

According to our database, Nolan Dey authored at least 10 papers between 2020 and 2025.

Collaborative distances:

• Dijkstra number of five.
• Erdős number of four.

2025

Power Lines: Scaling Laws for Weight Decay and Batch Size in LLM Pre-training.

Shane Bergsma, Nolan Dey, Gurpreet Gosal, Gavia Gray, Daria Soboleva, Joel Hestness

CoRR, May, 2025

Don't be lazy: CompleteP enables compute-efficient deep transformers.

Nolan Dey, Bin Claire Zhang, Lorenzo Noci, Mufan Bill Li, Blake Bordelon, Shane Bergsma, Cengiz Pehlevan, Boris Hanin, Joel Hestness

CoRR, May, 2025

Neuron-based explanations of neural networks sacrifice completeness and interpretability.

Nolan Simran Dey, J. Eric Taylor, Alexander Wong, Bryan P. Tripp, Graham W. Taylor

Trans. Mach. Learn. Res., 2025

Straight to Zero: Why Linearly Decaying the Learning Rate to Zero Works Best for LLMs.

Shane Bergsma, Nolan Simran Dey, Gurpreet Gosal, Gavia Gray, Daria Soboleva, Joel Hestness

Proceedings of the Thirteenth International Conference on Learning Representations, 2025

2024

Sparse maximal update parameterization: A holistic approach to sparse training dynamics.

Nolan Dey, Shane Bergsma, Joel Hestness

Proceedings of the Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, 2024

2023

Position Interpolation Improves ALiBi Extrapolation.

Faisal Al-Khateeb, Nolan Dey, Daria Soboleva, Joel Hestness

CoRR, 2023

BTLM-3B-8K: 7B Parameter Performance in a 3B Parameter Model.

Nolan Dey, Daria Soboleva, Faisal Al-Khateeb, Bowen Yang, Ribhu Pathria, Hemant Khachane, Shaheer Muhammad, Zhiming Chen, Robert Myers, Jacob Robert Steeves, Natalia Vassilieva, Marvin Tom, Joel Hestness

CoRR, 2023

Cerebras-GPT: Open Compute-Optimal Language Models Trained on the Cerebras Wafer-Scale Cluster.

Nolan Dey, Gurpreet Gosal, Zhiming Chen, Hemant Khachane, William Marshall, Ribhu Pathria, Marvin Tom, Joel Hestness

CoRR, 2023

2020

37, 000 Human-Planned Robotic Grasps With Six Degrees of Freedom.

Victor Reyes Osorio, Rajan Iyengar, Xueyang Yao, Presish Bhattachan, Adrian Ragobar, Nolan Dey, Bryan P. Tripp

IEEE Robotics Autom. Lett., 2020

Identifying and interpreting tuning dimensions in deep networks.

Nolan S. Dey, J. Eric Taylor, Bryan P. Tripp, Alexander Wong, Graham W. Taylor

CoRR, 2020