Beidi Chen

According to our database, Beidi Chen authored at least 48 papers between 2016 and 2024.

Collaborative distances:

• Dijkstra number of three.
• Erdős number of three.

2024

Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding.

Zhenyu Zhang, Runjin Chen, Shiwei Liu, Zhewei Yao, Olatunji Ruwase, Beidi Chen, Xiaoxia Wu, Zhangyang Wang

CoRR, 2024

GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection.

Jiawei Zhao, Zhenyu Zhang, Beidi Chen, Zhangyang Wang, Anima Anandkumar, Yuandong Tian

CoRR, 2024

LLM Inference Unveiled: Survey and Roofline Model Insights.

Zhihang Yuan, Yuzhang Shang, Yang Zhou, Zhen Dong, Zhe Zhou, Chenhao Xue, Bingzhe Wu, Zhikai Li, Qingyi Gu, Yong Jae Lee, Yan Yan, Beidi Chen, Guangyu Sun, Kurt Keutzer

CoRR, 2024

Sequoia: Scalable, Robust, and Hardware-aware Speculative Decoding.

Zhuoming Chen, Avner May, Ruslan Svirschevski, Yuhsun Huang, Max Ryabinin, Zhihao Jia, Beidi Chen

CoRR, 2024

Get More with LESS: Synthesizing Recurrence with KV Cache Compression for Efficient LLM Inference.

Harry Dong, Xinyu Yang, Zhenyu Zhang, Zhangyang Wang, Yuejie Chi, Beidi Chen

CoRR, 2024

Learn To be Efficient: Build Structured Sparsity in Large Language Models.

Haizhong Zheng, Xiaoyan Bai, Beidi Chen, Fan Lai, Atul Prakash

CoRR, 2024

KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache.

Zirui Liu, Jiayi Yuan, Hongye Jin, Shaochen Zhong, Zhaozhuo Xu, Vladimir Braverman, Beidi Chen, Xia Hu

CoRR, 2024

2023

HexGen: Generative Inference of Foundation Model over Heterogeneous Decentralized Environment.

Youhe Jiang, Ran Yan, Xiaozhe Yao, Beidi Chen, Binhang Yuan

CoRR, 2023

JoMA: Demystifying Multilayer Transformers via JOint Dynamics of MLP and Attention.

Yuandong Tian, Yiping Wang, Zhenyu Zhang, Beidi Chen, Simon S. Du

CoRR, 2023

Efficient Streaming Language Models with Attention Sinks.

Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, Mike Lewis

CoRR, 2023

H₂O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models.

Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Ré, Clark W. Barrett, Zhangyang Wang, Beidi Chen

CoRR, 2023

InRank: Incremental Low-Rank Learning.

Jiawei Zhao, Yifei Zhang, Beidi Chen, Florian Schäfer, Anima Anandkumar

CoRR, 2023

Compress, Then Prompt: Improving Accuracy-Efficiency Trade-off of LLM Inference with Transferable Prompt.

Zhaozhuo Xu, Zirui Liu, Beidi Chen, Yuxin Tang, Jue Wang, Kaixiong Zhou, Xia Hu, Anshumali Shrivastava

CoRR, 2023

High-throughput Generative Inference of Large Language Models with a Single GPU.

Ying Sheng, Lianmin Zheng, Binhang Yuan, Zhuohan Li, Max Ryabinin, Daniel Y. Fu, Zhiqiang Xie, Beidi Chen, Clark W. Barrett, Joseph E. Gonzalez, Percy Liang, Christopher Ré, Ion Stoica, Ce Zhang

CoRR, 2023

Modeling Scattering Coefficients using Self-Attentive Complex Polynomials with Image-based Representation.

Andrew Cohen, Weiping Dou, Jiang Zhu, Slawomir Koziel, Peter Renner, Jan-Ove Mattsson, Xiaomeng Yang, Beidi Chen, Kevin Stone, Yuandong Tian

CoRR, 2023

H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models.

Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Ré, Clark W. Barrett, Zhangyang Wang, Beidi Chen

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer.

Yuandong Tian, Yiping Wang, Beidi Chen, Simon S. Du

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

Laughing Hyena Distillery: Extracting Compact Recurrences From Convolutions.

Stefano Massaroli, Michael Poli, Daniel Y. Fu, Hermann Kumbong, Rom N. Parnichkun, David W. Romero, Aman Timalsina, Quinn McIntyre, Beidi Chen, Atri Rudra, Ce Zhang, Christopher Ré, Stefano Ermon, Yoshua Bengio

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

CocktailSGD: Fine-tuning Foundation Models over 500Mbps Networks.

Jue Wang, Yucheng Lu, Binhang Yuan, Beidi Chen, Percy Liang, Christopher De Sa, Christopher Ré, Ce Zhang

Proceedings of the International Conference on Machine Learning, 2023

Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time.

Zichang Liu, Jue Wang, Tri Dao, Tianyi Zhou, Binhang Yuan, Zhao Song, Anshumali Shrivastava, Ce Zhang, Yuandong Tian, Christopher Ré, Beidi Chen

Proceedings of the International Conference on Machine Learning, 2023

FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU.

Ying Sheng, Lianmin Zheng, Binhang Yuan, Zhuohan Li, Max Ryabinin, Beidi Chen, Percy Liang, Christopher Ré, Ion Stoica, Ce Zhang

Proceedings of the International Conference on Machine Learning, 2023

Fast Algorithms for a New Relaxation of Optimal Transport.

Moses Charikar, Beidi Chen, Christopher Ré, Erik Waingarten

Proceedings of the Thirty Sixth Annual Conference on Learning Theory, 2023

2022

Fine-tuning Language Models over Slow Networks using Activation Compression with Guarantees.

Jue Wang, Binhang Yuan, Luka Rimanic, Yongjun He, Tri Dao, Beidi Chen, Christopher Ré, Ce Zhang

CoRR, 2022

Decentralized Training of Foundation Models in Heterogeneous Environments.

Binhang Yuan, Yongjun He, Jared Davis, Tianyi Zhang, Tri Dao, Beidi Chen, Percy Liang, Christopher Ré, Ce Zhang

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

Fine-tuning Language Models over Slow Networks using Activation Quantization with Guarantees.

Jue Wang, Binhang Yuan, Luka Rimanic, Yongjun He, Tri Dao, Beidi Chen, Christopher Ré, Ce Zhang

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

HALOS: Hashing Large Output Space for Cheap Inference.

Zichang Liu, Zhaozhuo Xu, Alan Baonan Ji, Junyan Zhang, Jonathan Li, Beidi Chen, Anshumali Shrivastava

Proceedings of Machine Learning and Systems 2022, 2022

Monarch: Expressive Structured Matrices for Efficient and Accurate Training.

Tri Dao, Beidi Chen, Nimit Sharad Sohoni, Arjun D. Desai, Michael Poli, Jessica Grogan, Alexander Liu, Aniruddh Rao, Atri Rudra, Christopher Ré

Proceedings of the International Conference on Machine Learning, 2022

Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models.

Beidi Chen, Tri Dao, Kaizhao Liang, Jiaming Yang, Zhao Song, Atri Rudra, Christopher Ré

Proceedings of the Tenth International Conference on Learning Representations, 2022

2021

Satellite Images and Deep Learning to Identify Discrepancy in Mailing Addresses with Applications to Census 2020 in Houston.

Zhaozhuo Xu, Alan Baonan Ji, Andrew Woods, Beidi Chen, Anshumali Shrivastava

CoRR, 2021

Scatterbrain: Unifying Sparse and Low-rank Attention Approximation.

Beidi Chen, Tri Dao, Eric Winsor, Zhao Song, Atri Rudra, Christopher Ré

CoRR, 2021

Locality Sensitive Teaching.

Zhaozhuo Xu, Beidi Chen, Chaojian Li, Weiyang Liu, Le Song, Yingyan Lin, Anshumali Shrivastava

Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, 2021

Scatterbrain: Unifying Sparse and Low-rank Attention.

Beidi Chen, Tri Dao, Eric Winsor, Zhao Song, Atri Rudra, Christopher Ré

Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, 2021

A Tale of Two Efficient and Informative Negative Sampling Distributions.

Shabnam Daghaghi, Tharun Medini, Nicholas Meisburger, Beidi Chen, Mengnan Zhao, Anshumali Shrivastava

Proceedings of the 38th International Conference on Machine Learning, 2021

SOLAR: Sparse Orthogonal Learned and Random Embeddings.

Tharun Medini, Beidi Chen, Anshumali Shrivastava

Proceedings of the 9th International Conference on Learning Representations, 2021

MONGOOSE: A Learnable LSH Framework for Efficient Neural Network Training.

Beidi Chen, Zichang Liu, Binghui Peng, Zhaozhuo Xu, Jonathan Lingjie Li, Tri Dao, Zhao Song, Anshumali Shrivastava, Christopher Ré

Proceedings of the 9th International Conference on Learning Representations, 2021

2020

A Constant-time Adaptive Negative Sampling.

Shabnam Daghaghi, Tharun Medini, Beidi Chen, Mengnan Zhao, Anshumali Shrivastava

CoRR, 2020

Climbing the WOL: Training for Cheaper Inference.

Zichang Liu, Zhaozhuo Xu, Alan Baonan Ji, Jonathan Li, Beidi Chen, Anshumali Shrivastava

CoRR, 2020

SLIDE : In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale Deep Learning Systems.

Beidi Chen, Tharun Medini, James Farwell, Sameh Gobriel, Tsung-Yuan Charlie Tai, Anshumali Shrivastava

Proceedings of Machine Learning and Systems 2020, 2020

Angular Visual Hardness.

Beidi Chen, Weiyang Liu, Zhiding Yu, Jan Kautz, Anshumali Shrivastava, Animesh Garg, Animashree Anandkumar

Proceedings of the 37th International Conference on Machine Learning, 2020

2019

Sub-Linear Privacy-Preserving Near-Neighbor Search.

M. Sadegh Riazi, Beidi Chen, Anshumali Shrivastava, Dan S. Wallach, Farinaz Koushanfar

IACR Cryptol. ePrint Arch., 2019

Lsh-sampling Breaks the Computation Chicken-and-egg Loop in Adaptive Stochastic Gradient Estimation.

Beidi Chen, Yingchen Xu, Anshumali Shrivastava

CoRR, 2019

SLIDE : In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale Deep Learning Systems.

Beidi Chen, Tharun Medini, Anshumali Shrivastava

CoRR, 2019

Fast and Accurate Stochastic Gradient Estimation.

Beidi Chen, Yingchen Xu, Anshumali Shrivastava

Proceedings of the Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, 2019

2018

Densified Winner Take All (WTA) Hashing for Sparse Datasets.

Beidi Chen, Anshumali Shrivastava

Proceedings of the Thirty-Fourth Conference on Uncertainty in Artificial Intelligence, 2018

Lsh-Sampling breaks the Computational chicken-and-egg Loop in adaptive stochastic Gradient estimation.

Beidi Chen, Yingchen Xu, Anshumali Shrivastava

Proceedings of the 6th International Conference on Learning Representations, 2018

2017

Unique Entity Estimation with Application to the Syrian Conflict.

Beidi Chen, Anshumali Shrivastava, Rebecca C. Steorts

CoRR, 2017

2016

Sub-linear Privacy-preserving Search with Untrusted Server and Semi-honest Parties.

M. Sadegh Riazi, Beidi Chen, Anshumali Shrivastava, Dan S. Wallach, Farinaz Koushanfar

CoRR, 2016

Revisiting Winner Take All (WTA) Hashing for Sparse Datasets.

Beidi Chen, Anshumali Shrivastava

CoRR, 2016