Kushal Tirumala

According to our database, Kushal Tirumala authored at least 9 papers between 2020 and 2024.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of three.

2024

The Unreasonable Ineffectiveness of the Deeper Layers.

Andrey Gromov, Kushal Tirumala, Hassan Shapourian, Paolo Glorioso, Daniel A. Roberts

CoRR, 2024

Effective pruning of web-scale datasets based on complexity of concept clusters.

Amro Abbas, Evgenia Rusak, Kushal Tirumala, Wieland Brendel, Kamalika Chaudhuri, Ari S. Morcos

CoRR, 2024

2023

Decoding Data Quality via Synthetic Corruptions: Embedding-guided Pruning of Code Data.

Yu Yang, Aaditya K. Singh, Mostafa Elhoushi, Anas Mahmoud, Kushal Tirumala, Fabian Gloeckle, Baptiste Rozière, Carole-Jean Wu, Ari S. Morcos, Newsha Ardalani

CoRR, 2023

SemDeDup: Data-efficient learning at web-scale through semantic deduplication.

Amro Abbas, Kushal Tirumala, Daniel Simig, Surya Ganguli, Ari S. Morcos

CoRR, 2023

D4: Improving LLM Pretraining via Document De-Duplication and Diversification.

Kushal Tirumala, Daniel Simig, Armen Aghajanyan, Ari Morcos

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

2022

Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models.

Kushal Tirumala, Aram H. Markosyan, Luke Zettlemoyer, Armen Aghajanyan

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

Investigating Generalization by Controlling Normalized Margin.

Alexander R. Farhang, Jeremy D. Bernstein, Kushal Tirumala, Yang Liu, Yisong Yue

Proceedings of the International Conference on Machine Learning, 2022

Dynatask: A Framework for Creating Dynamic AI Benchmark Tasks.

Tristan Thrush, Kushal Tirumala, Anmol Gupta, Max Bartolo, Pedro Rodriguez, Tariq Kane, William Gaviria Rojas, Peter Mattson, Adina Williams, Douwe Kiela

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, 2022

2020

Ensemble Machine Learning Methods for Modeling COVID19 Deaths.

Rahil Bathwal, Pavan Chitta, Kushal Tirumala, Vignesh Varadarajan

CoRR, 2020