SpecInfer: Accelerating Generative LLM Serving with Speculative Inference and Token Tree Verification. - Xupeng Miao; Gabriele Oliaro; Zhihao Zhang 001; Xinhao Cheng; Zeyu Wang; Rae Ying Yee Wong; Zhuoming Chen; Daiyaan Arfeen; Reyna Abhyankar; Zhihao Jia (CoRR, 2023)

SpecInfer: Accelerating Large Language Model Serving with Tree-based Speculative Inference and Verification. - Xupeng Miao; Gabriele Oliaro; Zhihao Zhang 001; Xinhao Cheng; Zeyu Wang; Zhengxin Zhang; Rae Ying Yee Wong; Alan Zhu 001; Lijie Yang 003; Xiaoxiang Shi; Chunan Shi; Zhuoming Chen; et al. (ASPLOS (3), 2024)