Adaptive Braking for Mitigating Gradient Delay. - Abhinav Venigalla; Atli Kosson; Vitaliy Chiley; Urs Köster (CoRR, 2020)

Pipelined Backpropagation at Scale: Training Large Models without Batches. - Atli Kosson; Vitaliy Chiley; Abhinav Venigalla; Joel Hestness; Urs Köster (MLSys, 2021)

Representation range needs for 16-bit neural network training. - Valentina Popescu; Abhinav Venigalla; Di Wu; Robert Schreiber (CoRR, 2021)

MosaicBERT: A Bidirectional Encoder Optimized for Fast Pretraining. - Jacob P. Portes; Alexander Trott; Sam Havens; Daniel King; Abhinav Venigalla; Moin Nadeem; Nikhil Sardana; Daya Khudia; Jonathan Frankle (NeurIPS, 2023)

BioMedLM: A 2.7B Parameter Language Model Trained On Biomedical Text. - Elliot Bolton; Abhinav Venigalla; Michihiro Yasunaga; David Hall 006; Betty Xiong; Tony Lee; Roxana Daneshjou; Jonathan Frankle; Percy Liang; Michael Carbin; Christopher D. Manning (CoRR, 2024)