Revisiting Winner Take All (WTA) Hashing for Sparse Datasets. - Beidi Chen; Anshumali Shrivastava (CoRR, 2016)

Sub-linear Privacy-preserving Search with Untrusted Server and Semi-honest Parties. - M. Sadegh Riazi; Beidi Chen; Anshumali Shrivastava; Dan S. Wallach; Farinaz Koushanfar (CoRR, 2016)

Unique Entity Estimation with Application to the Syrian Conflict. - Beidi Chen; Anshumali Shrivastava; Rebecca C. Steorts (CoRR, 2017)

Lsh-Sampling breaks the Computational chicken-and-egg Loop in adaptive stochastic Gradient estimation. - Beidi Chen; Yingchen Xu; Anshumali Shrivastava (ICLR (Workshop), 2018)

Densified Winner Take All (WTA) Hashing for Sparse Datasets. - Beidi Chen; Anshumali Shrivastava (UAI, 2018)

Fast and Accurate Stochastic Gradient Estimation. - Beidi Chen; Yingchen Xu; Anshumali Shrivastava (NeurIPS, 2019)

SLIDE : In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale Deep Learning Systems. - Beidi Chen; Tharun Medini; Anshumali Shrivastava (CoRR, 2019)

Lsh-sampling Breaks the Computation Chicken-and-egg Loop in Adaptive Stochastic Gradient Estimation. - Beidi Chen; Yingchen Xu; Anshumali Shrivastava (CoRR, 2019)

Sub-Linear Privacy-Preserving Near-Neighbor Search. - M. Sadegh Riazi; Beidi Chen; Anshumali Shrivastava; Dan S. Wallach; Farinaz Koushanfar (IACR Cryptol. ePrint Arch., 2019)

Angular Visual Hardness. - Beidi Chen; Weiyang Liu; Zhiding Yu; Jan Kautz; Anshumali Shrivastava; Animesh Garg; Animashree Anandkumar (ICML, 2020)

SLIDE : In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale Deep Learning Systems. - Beidi Chen; Tharun Medini; James Farwell; Sameh Gobriel; Tsung-Yuan Charlie Tai; Anshumali Shrivastava (MLSys, 2020)

Climbing the WOL: Training for Cheaper Inference. - Zichang Liu; Zhaozhuo Xu; Alan Baonan Ji; Jonathan Li 002; Beidi Chen; Anshumali Shrivastava (CoRR, 2020)

A Constant-time Adaptive Negative Sampling. - Shabnam Daghaghi; Tharun Medini; Beidi Chen; Mengnan Zhao; Anshumali Shrivastava (CoRR, 2020)

MONGOOSE: A Learnable LSH Framework for Efficient Neural Network Training. - Beidi Chen; Zichang Liu; Binghui Peng; Zhaozhuo Xu; Jonathan Lingjie Li; Tri Dao; Zhao Song 002; Anshumali Shrivastava; Christopher Ré (ICLR, 2021)

SOLAR: Sparse Orthogonal Learned and Random Embeddings. - Tharun Medini; Beidi Chen; Anshumali Shrivastava (ICLR, 2021)

A Tale of Two Efficient and Informative Negative Sampling Distributions. - Shabnam Daghaghi; Tharun Medini; Nicholas Meisburger; Beidi Chen; Mengnan Zhao; Anshumali Shrivastava (ICML, 2021)

Scatterbrain: Unifying Sparse and Low-rank Attention. - Beidi Chen; Tri Dao; Eric Winsor; Zhao Song 002; Atri Rudra; Christopher Ré (NeurIPS, 2021)

Locality Sensitive Teaching. - Zhaozhuo Xu; Beidi Chen; Chaojian Li; Weiyang Liu; Le Song; Yingyan Lin; Anshumali Shrivastava (NeurIPS, 2021)

Scatterbrain: Unifying Sparse and Low-rank Attention Approximation. - Beidi Chen; Tri Dao; Eric Winsor; Zhao Song 002; Atri Rudra; Christopher Ré (CoRR, 2021)

Satellite Images and Deep Learning to Identify Discrepancy in Mailing Addresses with Applications to Census 2020 in Houston. - Zhaozhuo Xu; Alan Baonan Ji; Andrew Woods; Beidi Chen; Anshumali Shrivastava (CoRR, 2021)

Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models. - Beidi Chen; Tri Dao; Kaizhao Liang; Jiaming Yang; Zhao Song 002; Atri Rudra; Christopher Ré (ICLR, 2022)

Monarch: Expressive Structured Matrices for Efficient and Accurate Training. - Tri Dao; Beidi Chen; Nimit Sharad Sohoni; Arjun D. Desai; Michael Poli; Jessica Grogan; Alexander Liu; Aniruddh Rao; Atri Rudra; Christopher Ré (ICML, 2022)

HALOS: Hashing Large Output Space for Cheap Inference. - Zichang Liu; Zhaozhuo Xu; Alan Baonan Ji; Junyan Zhang; Jonathan Li 002; Beidi Chen; Anshumali Shrivastava (MLSys, 2022)

Fine-tuning Language Models over Slow Networks using Activation Quantization with Guarantees. - Jue Wang; Binhang Yuan; Luka Rimanic; Yongjun He 004; Tri Dao; Beidi Chen; Christopher Ré; Ce Zhang 001 (NeurIPS, 2022)

Decentralized Training of Foundation Models in Heterogeneous Environments. - Binhang Yuan; Yongjun He 004; Jared Davis; Tianyi Zhang; Tri Dao; Beidi Chen; Percy Liang; Christopher Ré; Ce Zhang 001 (NeurIPS, 2022)

Fine-tuning Language Models over Slow Networks using Activation Compression with Guarantees. - Jue Wang; Binhang Yuan; Luka Rimanic; Yongjun He 004; Tri Dao; Beidi Chen; Christopher Ré; Ce Zhang 001 (CoRR, 2022)

Fast Algorithms for a New Relaxation of Optimal Transport. - Moses Charikar; Beidi Chen; Christopher Ré; Erik Waingarten (COLT, 2023)

FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU. - Ying Sheng 007; Lianmin Zheng; Binhang Yuan; Zhuohan Li 001; Max Ryabinin; Beidi Chen; Percy Liang; Christopher Ré; Ion Stoica; Ce Zhang 001 (ICML, 2023)

Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time. - Zichang Liu; Jue Wang; Tri Dao; Tianyi Zhou 002; Binhang Yuan; Zhao Song 002; Anshumali Shrivastava; Ce Zhang 001; Yuandong Tian; Christopher Ré; Beidi Chen (ICML, 2023)

CocktailSGD: Fine-tuning Foundation Models over 500Mbps Networks. - Jue Wang; Yucheng Lu; Binhang Yuan; Beidi Chen; Percy Liang; Christopher De Sa; Christopher Ré; Ce Zhang 001 (ICML, 2023)

Laughing Hyena Distillery: Extracting Compact Recurrences From Convolutions. - Stefano Massaroli; Michael Poli; Daniel Y. Fu; Hermann Kumbong; Rom N. Parnichkun; David W. Romero; Aman Timalsina; Quinn McIntyre; Beidi Chen; Atri Rudra; Ce Zhang 001; Christopher Ré; et al. (NeurIPS, 2023)

Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer. - Yuandong Tian; Yiping Wang; Beidi Chen; Simon S. Du (NeurIPS, 2023)

H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models. - Zhenyu Zhang 015; Ying Sheng 007; Tianyi Zhou 001; Tianlong Chen 001; Lianmin Zheng; Ruisi Cai; Zhao Song 002; Yuandong Tian; Christopher Ré; Clark W. Barrett; Zhangyang Wang; Beidi Chen (NeurIPS, 2023)

Modeling Scattering Coefficients using Self-Attentive Complex Polynomials with Image-based Representation. - Andrew Cohen; Weiping Dou; Jiang Zhu; Slawomir Koziel; Peter Renner; Jan-Ove Mattsson; Xiaomeng Yang; Beidi Chen; Kevin Stone; Yuandong Tian (CoRR, 2023)

High-throughput Generative Inference of Large Language Models with a Single GPU. - Ying Sheng 007; Lianmin Zheng; Binhang Yuan; Zhuohan Li 001; Max Ryabinin; Daniel Y. Fu; Zhiqiang Xie; Beidi Chen; Clark W. Barrett; Joseph E. Gonzalez; Percy Liang; Christopher Ré; et al. (CoRR, 2023)

Compress, Then Prompt: Improving Accuracy-Efficiency Trade-off of LLM Inference with Transferable Prompt. - Zhaozhuo Xu; Zirui Liu 001; Beidi Chen; Yuxin Tang; Jue Wang; Kaixiong Zhou; Xia Hu 001; Anshumali Shrivastava (CoRR, 2023)

InRank: Incremental Low-Rank Learning. - Jiawei Zhao; Yifei Zhang; Beidi Chen; Florian Schäfer 001; Anima Anandkumar (CoRR, 2023)

H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models. - Zhenyu Zhang 015; Ying Sheng 007; Tianyi Zhou 002; Tianlong Chen 001; Lianmin Zheng; Ruisi Cai; Zhao Song 002; Yuandong Tian; Christopher Ré; Clark W. Barrett; Zhangyang Wang; Beidi Chen (CoRR, 2023)

HexGen: Generative Inference of Foundation Model over Heterogeneous Decentralized Environment. - Youhe Jiang; Ran Yan; Xiaozhe Yao; Beidi Chen; Binhang Yuan (CoRR, 2023)

LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding. - Mostafa Elhoushi; Akshat Shrivastava; Diana Liskovich; Basil Hosmer; Bram Wasti; Liangzhen Lai; Anas Mahmoud 002; Bilge Acun; Saurabh Agarwal; Ahmed Roman; Ahmed A Aly; Beidi Chen; et al. (ACL (1), 2024)

JoMA: Demystifying Multilayer Transformers via Joint Dynamics of MLP and Attention. - Yuandong Tian; Yiping Wang; Zhenyu Zhang 015; Beidi Chen; Simon Shaolei Du (ICLR, 2024)

Efficient Streaming Language Models with Attention Sinks. - Guangxuan Xiao; Yuandong Tian; Beidi Chen; Song Han 003; Mike Lewis (ICLR, 2024)

LoCoCo: Dropping In Convolutions for Long Context Compression. - Ruisi Cai; Yuandong Tian; Zhangyang Wang; Beidi Chen (ICML, 2024)

Get More with LESS: Synthesizing Recurrence with KV Cache Compression for Efficient LLM Inference. - Harry Dong; Xinyu Yang 002; Zhenyu Zhang 015; Zhangyang Wang; Yuejie Chi; Beidi Chen (ICML, 2024)

HexGen: Generative Inference of Large Language Model over Heterogeneous Environment. - Youhe Jiang; Ran Yan; Xiaozhe Yao; Yang Zhou; Beidi Chen; Binhang Yuan (ICML, 2024)

KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache. - Zirui Liu 001; Jiayi Yuan 001; Hongye Jin; Shaochen (Henry) Zhong; Zhaozhuo Xu; Vladimir Braverman; Beidi Chen; Xia Hu 001 (ICML, 2024)

Soft Prompt Recovers Compressed LLMs, Transferably. - Zhaozhuo Xu; Zirui Liu 001; Beidi Chen; Shaochen (Henry) Zhong; Yuxin Tang; Jue Wang; Kaixiong Zhou; Xia Hu 001; Anshumali Shrivastava (ICML, 2024)

GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection. - Jiawei Zhao; Zhenyu Zhang 015; Beidi Chen; Zhangyang Wang; Anima Anandkumar; Yuandong Tian (ICML, 2024)

Q-Hitter: A Better Token Oracle for Efficient LLM Inference via Sparse-Quantized KV Cache. - Zhenyu Zhang 015; Shiwei Liu 003; Runjin Chen; Bhavya Kailkhura; Beidi Chen; Atlas Wang (MLSys, 2024)

Federated Black-box Prompt Tuning System for Large Language Models on the Edge. - Yiming Li; Jingwei Sun 002; Yudong Liu; Yuandong Zhang; Ang Li 005; Beidi Chen; Holger R. Roth; Daguang Xu; Tingjun Chen; Yiran Chen 001 (MobiCom, 2024)

Nearest Neighbor Speculative Decoding for LLM Generation and Attribution. - Minghan Li 002; Xilun Chen 002; Ari Holtzman; Beidi Chen; Jimmy Lin; Scott Yih; Victoria Lin 002 (NeurIPS, 2024)

Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding. - Zhenyu Zhang 015; Runjin Chen; Shiwei Liu 003; Zhewei Yao; Olatunji Ruwase; Beidi Chen; Xiaoxia Wu; Zhangyang Wang (NeurIPS, 2024)

Sequoia: Scalable and Robust Speculative Decoding. - Zhuoming Chen; Avner May; Ruslan Svirschevski; Yuhsun Huang; Max Ryabinin; Zhihao Jia; Beidi Chen (NeurIPS, 2024)

On the Surprising Effectiveness of Attention Transfer for Vision Transformers. - Alexander C. Li; Yuandong Tian; Beidi Chen; Deepak Pathak; Xinlei Chen (NeurIPS, 2024)

Mini-Sequence Transformers: Optimizing Intermediate Memory for Long Sequences Training. - Cheng Luo; Jiawei Zhao; Zhuoming Chen; Beidi Chen; Animashree Anandkumar (NeurIPS, 2024)

Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length. - Xuezhe Ma; Xiaomeng Yang; Wenhan Xiong; Beidi Chen; Lili Yu; Hao Zhang 025; Jonathan May; Luke Zettlemoyer; Omer Levy; Chunting Zhou (NeurIPS, 2024)

SpecExec: Massively Parallel Speculative Decoding For Interactive LLM Inference on Consumer Devices. - Ruslan Svirschevski; Avner May; Zhuoming Chen; Beidi Chen; Zhihao Jia; Max Ryabinin (NeurIPS, 2024)

S2FT: Efficient, Scalable and Generalizable LLM Fine-tuning by Structured Sparsity. - Xinyu Yang; Jixuan Leng; Geyang Guo; Jiawei Zhao; Ryumei Nakada; Linjun Zhang; Huaxiu Yao; Beidi Chen (NeurIPS, 2024)

Model-GLUE: Democratized LLM Scaling for A Large Model Zoo in the Wild. - Xinyu Zhao; Guoheng Sun; Ruisi Cai; Yukun Zhou; Pingzhi Li; Peihao Wang; Bowen Tan; Yexiao He; Li Chen; Yi Liang; Beidi Chen; Binhang Yuan; et al. (NeurIPS, 2024)

Learn To be Efficient: Build Structured Sparsity in Large Language Models. - Haizhong Zheng; Xiaoyan Bai; Xueshen Liu; Zhuoqing Morley Mao; Beidi Chen; Fan Lai 001; Atul Prakash 001 (NeurIPS, 2024)

SIRIUS : Contexual Sparisty with Correction for Efficient LLMs. - Yang Zhou; Zhuoming Chen; Zhaozhuo Xu; Victoria Lin 002; Beidi Chen (NeurIPS, 2024)

Learn To be Efficient: Build Structured Sparsity in Large Language Models. - Haizhong Zheng; Xiaoyan Bai; Beidi Chen; Fan Lai 001; Atul Prakash 001 (CoRR, 2024)

Sequoia: Scalable, Robust, and Hardware-aware Speculative Decoding. - Zhuoming Chen; Avner May; Ruslan Svirschevski; Yuhsun Huang; Max Ryabinin; Zhihao Jia; Beidi Chen (CoRR, 2024)

LLM Inference Unveiled: Survey and Roofline Model Insights. - Zhihang Yuan; Yuzhang Shang; Yang Zhou; Zhen Dong 003; Zhe Zhou 002; Chenhao Xue; Bingzhe Wu; Zhikai Li; Qingyi Gu; Yong Jae Lee; Yan Yan 002; Beidi Chen; et al. (CoRR, 2024)

Prompt-prompted Mixture of Experts for Efficient LLM Generation. - Harry Dong; Beidi Chen; Yuejie Chi (CoRR, 2024)

TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding. - Hanshi Sun; Zhuoming Chen; Xinyu Yang 002; Yuandong Tian; Beidi Chen (CoRR, 2024)

Zeroth-Order Fine-Tuning of LLMs with Extreme Sparsity. - Wentao Guo; Jikai Long; Yimeng Zeng; Zirui Liu 001; Xinyu Yang 002; Yide Ran; Jacob R. Gardner; Osbert Bastani; Christopher De Sa; Xiaodong Yu; Beidi Chen; Zhaozhuo Xu (CoRR, 2024)

It Takes Two: On the Seamlessness between Reward and Policy Model in RLHF. - Taiming Lu; Lingfeng Shen; Xinyu Yang 002; Weiting Tan; Beidi Chen; Huaxiu Yao (CoRR, 2024)

VcLLM: Video Codecs are Secretly Tensor Codecs. - Ceyu Xu; Yongji Wu; Xinyu Yang 002; Beidi Chen; Matthew Lentz; Danyang Zhuo; Lisa Wu Wills (CoRR, 2024)

MINI-SEQUENCE TRANSFORMER: Optimizing Intermediate Memory for Long Sequences Training. - Cheng Luo; Jiawei Zhao; Zhuoming Chen; Beidi Chen; Anima Anandkumar (CoRR, 2024)

MagicDec: Breaking the Latency-Throughput Tradeoff for Long Context Generation with Speculative Decoding. - Jian Chen; Vashisth Tiwari; Ranajoy Sadhukhan; Zhuoming Chen; Jinyuan Shi; Ian En-Hsu Yen; Beidi Chen (CoRR, 2024)

Sirius: Contextual Sparsity with Correction for Efficient LLMs. - Yang Zhou; Zhuoming Chen; Zhaozhuo Xu; Victoria Lin 002; Beidi Chen (CoRR, 2024)

ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference. - Hanshi Sun; Li-Wen Chang; Wenlei Bao; Size Zheng 001; Ningxin Zheng; Xin Liu 086; Harry Dong; Yuejie Chi; Beidi Chen (CoRR, 2024)

APE: Faster and Longer Context-Augmented Generation via Adaptive Parallel Encoding. - Xinyu Yang 002; Tianqi Chen; Beidi Chen (ICLR, 2025)

MagicPIG: LSH Sampling for Efficient LLM Generation. - Zhuoming Chen; Ranajoy Sadhukhan; Zihao Ye; Yang Zhou; Jianyu Zhang; Niklas Nolte; Yuandong Tian; Matthijs Douze; Léon Bottou; Zhihao Jia; Beidi Chen (ICLR, 2025)

Zeroth-Order Fine-Tuning of LLMs with Transferable Static Sparsity. - Wentao Guo; Jikai Long; Yimeng Zeng; Zirui Liu 001; Xinyu Yang 002; Yide Ran; Jacob R. Gardner; Osbert Bastani; Christopher De Sa; Xiaodong Yu; Beidi Chen; Zhaozhuo Xu (ICLR, 2025)

MagicDec: Breaking the Latency-Throughput Tradeoff for Long Context Generation with Speculative Decoding. - Ranajoy Sadhukhan; Jian Chen; Zhuoming Chen; Vashisth Tiwari; Ruihang Lai; Jinyuan Shi; Ian En-Hsu Yen; Avner May; Tianqi Chen; Beidi Chen (ICLR, 2025)

Memory Mosaics. - Jianyu Zhang; Niklas Nolte; Ranajoy Sadhukhan; Beidi Chen; Léon Bottou (ICLR, 2025)

Speculative Prefill: Turbocharging TTFT with Lightweight and Training-Free Token Importance Estimation. - Jingyu Liu; Beidi Chen; Ce Zhang 001 (CoRR, 2025)

GSM-Infinite: How Do Your LLMs Behave over Infinitely Increasing Context Length and Reasoning Complexity? - Yang Zhou; Hongyi Liu; Zhuoming Chen; Yuandong Tian; Beidi Chen (CoRR, 2025)

HeadInfer: Memory-Efficient LLM Inference by Head-wise Offloading. - Cheng Luo; Zefan Cai; Hanshi Sun; Jinqi Xiao; Bo Yuan 001; Wen Xiao; Junjie Hu; Jiawei Zhao; Beidi Chen; Anima Anandkumar (CoRR, 2025)

Scalable LLM Math Reasoning Acceleration with Low-rank Distillation. - Harry Dong; Bilge Acun; Beidi Chen; Yuejie Chi (CoRR, 2025)

Act Only When It Pays: Efficient Reinforcement Learning for LLM Reasoning via Selective Rollouts. - Haizhong Zheng; Yang Zhou; Brian R. Bartoldson; Bhavya Kailkhura; Fan Lai 001; Jiawei Zhao; Beidi Chen (CoRR, 2025)

Kinetics: Rethinking Test-Time Scaling Laws. - Ranajoy Sadhukhan; Zhuoming Chen; Haizhong Zheng; Yang Zhou; Emma Strubell; Beidi Chen (CoRR, 2025)

Multiverse: Your Language Models Secretly Decide How to Parallelize and Merge Generation. - Xinyu Yang; Yuwei An; Hongyi Liu; Tianqi Chen; Beidi Chen (CoRR, 2025)