Flatter, faster: scaling momentum for optimal speedup of SGD. - Aditya Cowsik; Tankut Can; Paolo Glorioso (CoRR, 2022)

BlackMamba: Mixture of Experts for State-Space Models. - Quentin Anthony; Yury Tokpanov; Paolo Glorioso; Beren Millidge (CoRR, 2024)

Zamba: A Compact 7B SSM Hybrid Model. - Paolo Glorioso; Quentin Anthony; Yury Tokpanov; James Whittington; Jonathan Pilault; Adam Ibrahim; Beren Millidge (CoRR, 2024)

Zyda: A 1.3T Dataset for Open Language Modeling. - Yury Tokpanov; Beren Millidge; Paolo Glorioso; Jonathan Pilault; Adam Ibrahim; James Whittington; Quentin Anthony (CoRR, 2024)

Zyda-2: a 5 Trillion Token High-Quality Dataset. - Yury Tokpanov; Paolo Glorioso; Quentin Anthony; Beren Millidge (CoRR, 2024)

The Zamba2 Suite: Technical Report. - Paolo Glorioso; Quentin Anthony; Yury Tokpanov; Anna Golubeva; Vasudev Shyam; James Whittington; Jonathan Pilault; Beren Millidge (CoRR, 2024)

The Unreasonable Ineffectiveness of the Deeper Layers. - Andrey Gromov; Kushal Tirumala; Hassan Shapourian; Paolo Glorioso; Daniel A. Roberts (ICLR, 2025)