Ben Athiwaratkun

According to our database, Ben Athiwaratkun authored at least 48 papers between 2015 and 2026.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of three.

2026

SAW-INT4: System-Aware 4-Bit KV-Cache Quantization for Real-World LLM Serving.

Jinda Jia, Jisen Li, Zhongzhu Zhou, Jung Hwan Heo, Jue Wang, Tri Dao, Shuaiwen Leon Song, Ben Athiwaratkun, Chenfeng Xu, Tianyi Zhang, Xiaoxia Wu

CoRR, April, 2026

Introspective Diffusion Language Models.

Yifan Yu, Yuqing Jian, Junxiong Wang, Zhongzhu Zhou, Donglin Zhuang, Xinyu Fang, Sri Yanamandra, Xiaoxia Wu, Qingyang Wu, Shuaiwen Leon Song, Tri Dao, Ben Athiwaratkun, James Zou, Fan Lai, Chenfeng Xu

CoRR, April, 2026

Squeeze Evolve: Unified Multi-Model Orchestration for Verifier-Free Evolution.

Monishwaran Maheswaran, Leon Lakhani, Zhongzhu Zhou, Shijia Yang, Junxiong Wang, Coleman Hooper, Yuezhou Hu, Rishabh Tiwari, Jue Wang, Harman Singh, Qingyang Wu, Yuqing Jian, Ce Zhang, Kurt Keutzer, Tri Dao, Xiaoxia Wu, Ben Athiwaratkun, James Zou, Chenfeng Xu

CoRR, April, 2026

CARE: Covariance-Aware and Rank-Enhanced Decomposition for Enabling Multi-Head Latent Attention.

Zhongzhu Zhou, Fengxiang Bie, Ziyan Chen, Zhenyu Zhang, Yibo Yang, Junxiong Wang, Ben Athiwaratkun, Xiaoxia Wu, Shuaiwen Leon Song

CoRR, March, 2026

V₁: Unifying Generation and Self-Verification for Parallel Reasoners.

Harman Singh, Xiuyu Li, Kusha Sareen, Monishwaran Maheswaran, Sijun Tan, Xiaoxia Wu, Junxiong Wang, Alpay Ariyak, Qingyang Wu, Samir Khaki, Rishabh Tiwari, Long Lian, Yucheng Lu, Boyi Li, Alane Suhr, Ben Athiwaratkun, Kurt Keutzer

CoRR, March, 2026

When RL Meets Adaptive Speculative Training: A Unified Training-Serving System.

Junxiong Wang, Fengxiang Bie, Jisen Li, Zhongzhu Zhou, Zelei Shao, Yubo Wang, Yinghui Liu, Qingyang Wu, Avner May, Sri Yanamandra, Yineng Zhang, Ce Zhang, Tri Dao, Percy Liang, Ben Athiwaratkun, Shuaiwen Leon Song, Chenfeng Xu, Xiaoxia Wu

CoRR, February, 2026

2025

Understanding and Steering the Cognitive Behaviors of Reasoning Models at Test-Time.

Zhenyu Zhang, Xiaoxia Wu, Zhongzhu Zhou, Qingyang Wu, Yineng Zhang, Pragaash Ponnusamy, Harikaran Subbaraj, Jue Wang, Shuaiwen Leon Song, Ben Athiwaratkun

CoRR, December, 2025

CDLM: Consistency Diffusion Language Models For Faster Sampling.

Minseo Kim, Chenfeng Xu, Coleman Hooper, Harman Singh, Ben Athiwaratkun, Ce Zhang, Kurt Keutzer, Amir Gholami

CoRR, November, 2025

Kitty: Accurate and Efficient 2-bit KV Cache Quantization with Dynamic Channel-wise Precision Boost.

Haojun Xia, Xiaoxia Wu, Jisen Li, Robert Wu, Junxiong Wang, Jue Wang, Chenxi Li, Aman Singhal, Alay Dilipbhai Shah, Alpay Ariyak, Donglin Zhuang, Zhongzhu Zhou, Ben Athiwaratkun, Zhen Zheng, Shuaiwen Leon Song

CoRR, November, 2025

Beat the long tail: Distribution-Aware Speculative Decoding for RL Training.

Zelei Shao, Vikranth Srivatsa, Sanjana Srivastava, Qingyang Wu, Alpay Ariyak, Xiaoxia Wu, Ameen Patel, Jue Wang, Percy Liang, Tri Dao, Ce Zhang, Yiying Zhang, Ben Athiwaratkun, Chenfeng Xu, Junxiong Wang

CoRR, November, 2025

Intelligence per Watt: Measuring Intelligence Efficiency of Local AI.

Jon Saad-Falcon, Avanika Narayan, Hakki Orhun Akengin, J. Wes Griffin, Herumb Shandilya, Adrian Gamarra Lafuente, Medhya Goel, Rebecca Joseph, Shlok Natarajan, Etash Kumar Guha, Shang Zhu, Ben Athiwaratkun, John Hennessy, Azalia Mirhoseini, Christopher Ré

CoRR, November, 2025

Opportunistic Expert Activation: Batch-Aware Expert Routing for Faster Decode Without Retraining.

Costin-Andrei Oncescu, Qingyang Wu, Wai Tong Chung, Robert Wu, Bryan Gopal, Junxiong Wang, Tri Dao, Ben Athiwaratkun

CoRR, November, 2025

Staircase Streaming for Low-Latency Multi-Agent Inference.

Junlin Wang, Jue Wang, Zhen Xu, Ben Athiwaratkun, Bhuwan Dhingra, Ce Zhang, James Zou

CoRR, October, 2025

Imitate Optimal Policy: Prevail and Induce Action Collapse in Policy Gradient.

Zhongzhu Zhou, Yibo Yang, Ziyan Chen, Fengxiang Bie, Haojun Xia, Xiaoxia Wu, Robert Wu, Ben Athiwaratkun, Bernard Ghanem, Shuaiwen Leon Song

CoRR, September, 2025

Data Diversification Methods In Alignment Enhance Math Performance In LLMs.

Berkan Dokmeci, Qingyang Wu, Ben Athiwaratkun, Ce Zhang, Shuaiwen Leon Song, James Zou

CoRR, July, 2025

Shrinking the Generation-Verification Gap with Weak Verifiers.

Jon Saad-Falcon, Estefany Kelly Buchanan, Mayee F. Chen, Tzu-Heng Huang, Brendan McLaughlin, Tanvir Bhathal, Shang Zhu, Ben Athiwaratkun, Frederic Sala, Scott W. Linderman, Azalia Mirhoseini, Christopher Ré

CoRR, June, 2025

When Does Divide and Conquer Work for Long Context LLM? A Noise Decomposition Framework.

Zhen Xu, Shang Zhu, Jue Wang, Junlin Wang, Ben Athiwaratkun, Chi Wang, James Zou, Ce Zhang

CoRR, June, 2025

Disentangling Reasoning and Knowledge in Medical Large Language Models.

Rahul Thapa, Qingyang Wu, Kevin Wu, Harrison Zhang, Angela Zhang, Eric Wu, Haotian Ye, Suhana Bedi, Nevin Aresh, Joseph Boen, Shriya Reddy, Ben Athiwaratkun, Shuaiwen Leon Song, James Zou

CoRR, May, 2025

Improving Model Alignment Through Collective Intelligence of Open-Source LLMS.

Junlin Wang, Roy Xie, Shang Zhu, Jue Wang, Ben Athiwaratkun, Bhuwan Dhingra, Shuaiwen Leon Song, Ce Zhang, James Zou

CoRR, May, 2025

How Well Can General Vision-Language Models Learn Medicine By Watching Public Educational Videos?

Rahul Thapa, Andrew Li, Qingyang Wu, Bryan He, Yuki Sahashi, Christina Binder, Angela Zhang, Ben Athiwaratkun, Shuaiwen Leon Song, David Ouyang, James Zou

CoRR, April, 2025

Think Deep, Think Fast: Investigating Efficiency of Verifier-free Inference-time-scaling Methods.

Junlin Wang, Shang Zhu, Jon Saad-Falcon, Ben Athiwaratkun, Qingyang Wu, Jue Wang, Shuaiwen Leon Song, Ce Zhang, Bhuwan Dhingra, James Zou

CoRR, April, 2025

Ladder-Residual: Parallelism-Aware Architecture for Accelerating Large Model Inference with Communication Overlapping.

Muru Zhang, Mayank Mishra, Zhongzhu Zhou, William Brandon, Jue Wang, Yoon Kim, Jonathan Ragan-Kelley, Shuaiwen Leon Song, Ben Athiwaratkun, Tri Dao

Proceedings of the Forty-second International Conference on Machine Learning, 2025

Improving Model Alignment Through Collective Intelligence of Open-Source Models.

Junlin Wang, Roy Xie, Shang Zhu, Jue Wang, Ben Athiwaratkun, Bhuwan Dhingra, Shuaiwen Leon Song, Ce Zhang, James Zou

Proceedings of the Forty-second International Conference on Machine Learning, 2025

Mixture-of-Agents Enhances Large Language Model Capabilities.

Junlin Wang, Jue Wang, Ben Athiwaratkun, Ce Zhang, James Zou

Proceedings of the Thirteenth International Conference on Learning Representations, 2025

Training-Free Activation Sparsity in Large Language Models.

James Liu, Pragaash Ponnusamy, Tianle Cai, Han Guo, Yoon Kim, Ben Athiwaratkun

Proceedings of the Thirteenth International Conference on Learning Representations, 2025

Scaling Instruction-tuned LLMs to Million-token Contexts via Hierarchical Synthetic Data Generation.

Linda He, Jue Wang, Maurice Weber, Shang Zhu, Ben Athiwaratkun, Ce Zhang

Proceedings of the Thirteenth International Conference on Learning Representations, 2025

2024

Dragonfly: Multi-Resolution Zoom Supercharges Large Visual-Language Model.

Kezhen Chen, Rahul Thapa, Rahul Chalamala, Ben Athiwaratkun, Shuaiwen Leon Song, James Zou

CoRR, 2024

Token Alignment via Character Matching for Subword Completion.

Ben Athiwaratkun, Shiqi Wang, Mingyue Shang, Yuchen Tian, Zijian Wang, Sujan Kumar Gonugondla, Sanjay Krishna Gouda, Rob Kwiatkowski, Ramesh Nallapati, Bing Xiang

CoRR, 2024

RedPajama: an Open Dataset for Training Large Language Models.

Maurice Weber, Daniel Y. Fu, Quentin Anthony, Yonatan Oren, Shane Adams, Anton Alexandrov, Xiaozhong Lyu, Huu Nguyen, Xiaozhe Yao, Virginia Adams, Ben Athiwaratkun, Rahul Chalamala, Kezhen Chen, Max Ryabinin, Tri Dao, Percy Liang, Christopher Ré, Irina Rish, Ce Zhang

Proceedings of the Advances in Neural Information Processing Systems 37: Annual Conference on Neural Information Processing Systems 2024, 2024

Bifurcated Attention for Single-Context Large-Batch Sampling.

Ben Athiwaratkun, Sujan Kumar Gonugondla, Sanjay Krishna Gouda, Haifeng Qian, Hantian Ding, Qing Sun, Jun Wang, Jiacheng Guo, Liangfu Chen, Parminder Bhatia, Ramesh Nallapati, Sudipta Sengupta, Bing Xiang

Proceedings of the Forty-first International Conference on Machine Learning, 2024

Reasoning in Token Economies: Budget-Aware Evaluation of LLM Reasoning Strategies.

Junlin Wang, Siddhartha Jain, Dejiao Zhang, Baishakhi Ray, Varun Kumar, Ben Athiwaratkun

Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024

Token Alignment via Character Matching for Subword Completion.

Ben Athiwaratkun, Shiqi Wang, Mingyue Shang, Yuchen Tian, Zijian Wang, Sujan Kumar Gonugondla, Sanjay Krishna Gouda, Robert Kwiatkowski, Ramesh Nallapati, Parminder Bhatia, Bing Xiang

Proceedings of the Findings of the Association for Computational Linguistics, 2024

2023

Greener yet Powerful: Taming Large Code Generation Models with Quantization.

Xiaokai Wei, Sujan K. Gonugondla, Wasi Uddin Ahmad, Shiqi Wang, Baishakhi Ray, Haifeng Qian, Xiaopeng Li, Varun Kumar, Zijian Wang, Yuchen Tian, Qing Sun, Ben Athiwaratkun, Mingyue Shang, Murali Krishna Ramanathan, Parminder Bhatia, Bing Xiang

CoRR, 2023

Towards Greener Yet Powerful Code Generation via Quantization: An Empirical Study.

Xiaokai Wei, Sujan Kumar Gonugondla, Shiqi Wang, Wasi Uddin Ahmad, Baishakhi Ray, Haifeng Qian, Xiaopeng Li, Varun Kumar, Zijian Wang, Yuchen Tian, Qing Sun, Ben Athiwaratkun, Mingyue Shang, Murali Krishna Ramanathan, Parminder Bhatia, Bing Xiang

Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 2023

Multi-lingual Evaluation of Code Generation Models.

Ben Athiwaratkun, Sanjay Krishna Gouda, Zijian Wang, Xiaopeng Li, Yuchen Tian, Ming Tan, Wasi Uddin Ahmad, Shiqi Wang, Qing Sun, Mingyue Shang, Sujan Kumar Gonugondla, Hantian Ding, Varun Kumar, Nathan Fulton, Arash Farahani, Siddhartha Jain, Robert Giaquinto, Haifeng Qian, Murali Krishna Ramanathan, Ramesh Nallapati

Proceedings of the Eleventh International Conference on Learning Representations, 2023

2022

Multi-lingual Evaluation of Code Generation Models.

Ben Athiwaratkun, Sanjay Krishna Gouda, Zijian Wang, Xiaopeng Li, Yuchen Tian, Ming Tan, Wasi Uddin Ahmad, Shiqi Wang, Qing Sun, Mingyue Shang, Sujan Kumar Gonugondla, Hantian Ding, Varun Kumar, Nathan Fulton, Arash Farahani, Siddhartha Jain, Robert Giaquinto, Haifeng Qian, Murali Krishna Ramanathan, Ramesh Nallapati, Baishakhi Ray, Parminder Bhatia, Sudipta Sengupta, Dan Roth, Bing Xiang

CoRR, 2022

2021

Joint Text and Label Generation for Spoken Language Understanding.

Yang Li, Ben Athiwaratkun, Cícero Nogueira dos Santos, Bing Xiang

CoRR, 2021

Structured Prediction as Translation between Augmented Natural Languages.

Giovanni Paolini, Ben Athiwaratkun, Jason Krone, Jie Ma, Alessandro Achille, Rishita Anubhai, Cícero Nogueira dos Santos, Bing Xiang, Stefano Soatto

Proceedings of the 9th International Conference on Learning Representations, 2021

Generative Context Pair Selection for Multi-hop Question Answering.

Dheeru Dua, Cícero Nogueira dos Santos, Patrick Ng, Ben Athiwaratkun, Bing Xiang, Matt Gardner, Sameer Singh

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021

2020

Augmented Natural Language for Generative Sequence Labeling.

Ben Athiwaratkun, Cícero Nogueira dos Santos, Jason Krone, Bing Xiang

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, 2020

2019

There Are Many Consistent Explanations of Unlabeled Data: Why You Should Average.

Ben Athiwaratkun, Marc Finzi, Pavel Izmailov, Andrew Gordon Wilson

Proceedings of the 7th International Conference on Learning Representations, 2019

2018

Adversarial Deep Averaging Networks for Cross-Lingual Sentiment Classification.

Xilun Chen, Yu Sun, Ben Athiwaratkun, Claire Cardie, Kilian Q. Weinberger

Trans. Assoc. Comput. Linguistics, 2018

Improving Consistency-Based Semi-Supervised Learning with Weight Averaging.

Ben Athiwaratkun, Marc Finzi, Pavel Izmailov, Andrew Gordon Wilson

CoRR, 2018

Hierarchical Density Order Embeddings.

Ben Athiwaratkun, Andrew Gordon Wilson

Proceedings of the 6th International Conference on Learning Representations, 2018

Probabilistic FastText for Multi-Sense Word Embeddings.

Ben Athiwaratkun, Andrew Gordon Wilson, Anima Anandkumar

Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2018

2017

Malware classification with LSTM and GRU language models and a character-level CNN.

Ben Athiwaratkun, Jack W. Stokes

Proceedings of the 2017 IEEE International Conference on Acoustics, 2017

Multimodal Word Distributions.

Ben Athiwaratkun, Andrew Gordon Wilson

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 2017

2015

Feature Representation in Convolutional Neural Networks.

Ben Athiwaratkun, Keegan Kang

CoRR, 2015