Gavia Gray

According to our database, Gavia Gray authored at least 3 papers between 2024 and 2025.

Collaborative distances:

• Dijkstra number of five.
• Erdős number of five.

2025

Power Lines: Scaling laws for weight decay and batch size in LLM pre-training.

Shane Bergsma, Nolan Dey, Gurpreet Gosal, Gavia Gray, Daria Soboleva, Joel Hestness

Proceedings of the Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2025, 2025

Straight to Zero: Why Linearly Decaying the Learning Rate to Zero Works Best for LLMs.

Shane Bergsma, Nolan Simran Dey, Gurpreet Gosal, Gavia Gray, Daria Soboleva, Joel Hestness

Proceedings of the Thirteenth International Conference on Learning Representations, 2025

2024

Normalization Layer Per-Example Gradients are Sufficient to Predict Gradient Noise Scale in Transformers.

Gavia Gray, Aman Tiwari, Shane Bergsma, Joel Hestness

Proceedings of the Advances in Neural Information Processing Systems 37: Annual Conference on Neural Information Processing Systems 2024, 2024