Scaling Blockchains: Can Elected Committees Help? - Alon Benhaim; Brett Hemenway Falk; Gerry Tsoukalas (CoRR, 2021)

Foundation Transformers. - Hongyu Wang 009; Shuming Ma; Shaohan Huang; Li Dong 004; Wenhui Wang 003; Zhiliang Peng; Yu Wu; Payal Bajaj; Saksham Singhal; Alon Benhaim; Barun Patra; Zhun Liu; et al. (CoRR, 2022)

TorchScale: Transformers at Scale. - Shuming Ma; Hongyu Wang 009; Shaohan Huang; Wenhui Wang 003; Zewen Chi; Li Dong 004; Alon Benhaim; Barun Patra; Vishrav Chaudhary; Xia Song; Furu Wei (CoRR, 2022)

A Length-Extrapolatable Transformer. - Yutao Sun; Li Dong 004; Barun Patra; Shuming Ma; Shaohan Huang; Alon Benhaim; Vishrav Chaudhary; Xia Song; Furu Wei (ACL (1), 2023)

Magneto: A Foundation Transformer. - Hongyu Wang 009; Shuming Ma; Shaohan Huang; Li Dong 004; Wenhui Wang 003; Zhiliang Peng; Yu Wu; Payal Bajaj; Saksham Singhal; Alon Benhaim; Barun Patra; Zhun Liu; et al. (ICML, 2023)

Scaling Blockchains: Can Committee-Based Consensus Help? - Alon Benhaim; Brett Hemenway Falk; Gerry Tsoukalas (Manag. Sci., 2023)

On the Adaptation of Unlimiformer for Decoder-Only Transformers. - Kian Ahrabian; Alon Benhaim; Barun Patra; Jay Pujara; Saksham Singhal; Xia Song (LREC/COLING, 2024)

Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone. - Marah I Abdin; Sam Ade Jacobs; Ammar Ahmad Awan; Jyoti Aneja; Ahmed Awadallah 001; Hany Awadalla; Nguyen Bach; Amit Bahree; Arash Bakhtiari; Harkirat S. Behl; Alon Benhaim; Misha Bilenko; et al. (CoRR, 2024)

The Hitchhiker's Guide to Human Alignment with *PO. - Kian Ahrabian; Xihui Lin; Barun Patra; Vishrav Chaudhary; Alon Benhaim; Jay Pujara; Xia Song (CoRR, 2024)

Scaling Optimal LR Across Token Horizon. - Johan Bjorck; Alon Benhaim; Vishrav Chaudhary; Furu Wei; Xia Song (CoRR, 2024)

POROver: Improving Safety and Reducing Overrefusal in Large Language Models with Overgeneration and Preference Optimization. - Batuhan K. Karaman; Ishmam Zabir; Alon Benhaim; Vishrav Chaudhary; Mert R. Sabuncu; Xia Song (CoRR, 2024)

Scaling Laws for Multilingual Language Models. - Yifei He; Alon Benhaim; Barun Patra; Praneetha Vaddamanu; Sanchit Ahuja; Parul Chopra; Vishrav Chaudhary; Han Zhao 002; Xia Song (ACL (Findings), 2025)

Scaling Optimal LR Across Token Horizons. - Johan Bjorck; Alon Benhaim; Vishrav Chaudhary; Furu Wei; Xia Song (ICLR, 2025)

A Practical Analysis of Human Alignment with *PO. - Kian Ahrabian; Xihui Lin; Barun Patra; Vishrav Chaudhary; Alon Benhaim; Jay Pujara; Xia Song (NAACL (Findings), 2025)

Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs. - Abdelrahman Abouelenin; Atabak Ashfaq; Adam Atkinson; Hany Awadalla; Nguyen Bach; Jianmin Bao; Alon Benhaim; Martin Cai; Vishrav Chaudhary; Congcong Chen; Dong Chen 003; Dongdong Chen 001; et al. (CoRR, 2025)