HMC-TRAN: A Tensor-core Inspired Hierarchical Model Compression for Transformer-based DNNs on GPU. - Shaoyi Huang; Shiyang Chen 004; Hongwu Peng; Daniel Manu; Zhenglun Kong; Geng Yuan; Lei Yang 018; Shusen Wang; Hang Liu 001; Caiwen Ding (ACM Great Lakes Symposium on VLSI, 2021)

Co-Exploration of Graph Neural Network and Network-on-Chip Design Using AutoML. - Daniel Manu; Shaoyi Huang; Caiwen Ding; Lei Yang 018 (ACM Great Lakes Symposium on VLSI, 2021)

Accommodating Transformer onto FPGA: Coupling the Balanced Model Compression and FPGA-Implementation Optimization. - Panjie Qi; Yuhong Song; Hongwu Peng; Shaoyi Huang; Qingfeng Zhuge; Edwin Hsing-Mean Sha (ACM Great Lakes Symposium on VLSI, 2021)

Accelerating Framework of Transformer by Hardware Design and Model Compression Co-Optimization. - Panjie Qi; Edwin Hsing-Mean Sha; Qingfeng Zhuge; Hongwu Peng; Shaoyi Huang; Zhenglun Kong; Yuhong Song; Bingbing Li (ICCAD, 2021)

Accelerating Transformer-based Deep Learning Models on FPGAs using Column Balanced Block Pruning. - Hongwu Peng; Shaoyi Huang; Tong Geng; Ang Li 006; Weiwen Jiang; Hang Liu 001; Shusen Wang; Caiwen Ding (ISQED, 2021)

E.T.: re-thinking self-attention for transformer models on GPUs. - Shiyang Chen 004; Shaoyi Huang; Santosh Pandey; Bingbing Li; Guang R. Gao; Long Zheng 001; Caiwen Ding; Hang Liu 001 (SC, 2021)

Sparse Progressive Distillation: Resolving Overfitting under Pretrain-and-Finetune Paradigm. - Shaoyi Huang; Dongkuan Xu; Ian En-Hsu Yen; Sung-En Chang; Bingbing Li; Shiyang Chen 004; Mimi Xie; Hang Liu 001; Caiwen Ding (CoRR, 2021)

Sparse Progressive Distillation: Resolving Overfitting under Pretrain-and-Finetune Paradigm. - Shaoyi Huang; Dongkuan Xu; Ian En-Hsu Yen; Yijue Wang; Sung-En Chang; Bingbing Li; Shiyang Chen 004; Mimi Xie; Sanguthevar Rajasekaran; Hang Liu 001; Caiwen Ding (ACL (1), 2022)

Analyzing and Defending against Membership Inference Attacks in Natural Language Processing Classification. - Yijue Wang; Nuo Xu; Shaoyi Huang; Kaleel Mahmood; Dan Guo; Caiwen Ding; Wujie Wen; Sanguthevar Rajasekaran (IEEE Big Data, 2022)

A length adaptive algorithm-hardware co-design of transformer on FPGA through sparse attention and dynamic pipelining. - Hongwu Peng; Shaoyi Huang; Shiyang Chen 004; Bingbing Li; Tong Geng; Ang Li 006; Weiwen Jiang; Wujie Wen; Jinbo Bi; Hang Liu 001; Caiwen Ding (DAC, 2022)

CoDG-ReRAM: An Algorithm-Hardware Co-design to Accelerate Semi-Structured GNNs on ReRAM. - Yixuan Luo; Payman Behnam; Kiran Thorat; Zhuo Liu; Hongwu Peng; Shaoyi Huang; Shu Zhou; Omer Khan; Alexey Tumanov; Caiwen Ding; Tong Geng (ICCD, 2022)

Towards Sparsification of Graph Neural Networks. - Hongwu Peng; Deniz Gurevin; Shaoyi Huang; Tong Geng; Weiwen Jiang; Omer Khan; Caiwen Ding (ICCD, 2022)

An Automatic and Efficient BERT Pruning for Edge AI Systems. - Shaoyi Huang; Ning Liu 007; Yueying Liang; Hongwu Peng; Hongjia Li; Dongkuan Xu; Mimi Xie; Caiwen Ding (ISQED, 2022)

Efficient Traffic State Forecasting using Spatio-Temporal Network Dependencies: A Sparse Graph Neural Network Approach. - Bin Lei; Shaoyi Huang; Caiwen Ding; Monika Filipovska (CoRR, 2022)

Neurogenesis Dynamics-inspired Spiking Neural Network Training Acceleration. - Shaoyi Huang; Haowen Fang; Kaleel Mahmood; Bowen Lei; Nuo Xu; Bin Lei; Yue Sun; Dongkuan Xu; Wujie Wen; Caiwen Ding (DAC, 2023)

Dynamic Sparse Training via Balancing the Exploration-Exploitation Trade-off. - Shaoyi Huang; Bowen Lei; Dongkuan Xu; Hongwu Peng; Yue Sun; Mimi Xie; Caiwen Ding (DAC, 2023)

Accel-GCN: High-Performance GPU Accelerator Design for Graph Convolution Networks. - Xi Xie; Hongwu Peng; Amit Hasan 001; Shaoyi Huang; Jiahui Zhao; Haowen Fang; Wei Zhang; Tong Geng; Omer Khan; Caiwen Ding (ICCAD, 2023)

AutoReP: Automatic ReLU Replacement for Fast Private Network Inference. - Hongwu Peng; Shaoyi Huang; Tong Zhou 002; Yukui Luo; Chenghong Wang; Zigeng Wang; Jiahui Zhao; Xi Xie; Ang Li 006; Tony Geng; Kaleel Mahmood; Wujie Wen; et al. (ICCV, 2023)

Towards Lossless Head Pruning through Automatic Peer Distillation for Language Models. - Bingbing Li; Zigeng Wang; Shaoyi Huang; Mikhail A. Bragin; Ji Li 006; Caiwen Ding (IJCAI, 2023)

LinGCN: Structural Linearized Graph Convolutional Network for Homomorphically Encrypted Inference. - Hongwu Peng; Ran Ran; Yukui Luo; Jiahui Zhao; Shaoyi Huang; Kiran Thorat; Tong Geng; Chenghong Wang; Xiaolin Xu 001; Wujie Wen; Caiwen Ding (NeurIPS, 2023)

RRNet: Towards ReLU-Reduced Neural Network for Two-party Computation Based Private Inference. - Hongwu Peng; Shanglin Zhou; Yukui Luo; Nuo Xu; Shijin Duan; Ran Ran; Jiahui Zhao; Shaoyi Huang; Xi Xie; Chenghong Wang; Tong Geng; Wujie Wen; et al. (CoRR, 2023)

MaxK-GNN: Towards Theoretical Speed Limits for Accelerating Graph Neural Networks Training. - Hongwu Peng; Xi Xie; Kaustubh Shivdikar; Md Amit Hasan 001; Jiahui Zhao; Shaoyi Huang; Omer Khan; David R. Kaeli; Caiwen Ding (CoRR, 2023)

MaxK-GNN: Extremely Fast GPU Kernel Design for Accelerating Graph Neural Networks Training. - Hongwu Peng; Xi Xie; Kaustubh Shivdikar; Md Amit Hasan 001; Jiahui Zhao; Shaoyi Huang; Omer Khan; David R. Kaeli; Caiwen Ding (ASPLOS (2), 2024)

PruneGNN: Algorithm-Architecture Pruning Framework for Graph Neural Network Acceleration. - Deniz Gurevin; Mohsin Shan; Shaoyi Huang; Md Amit Hasan 001; Caiwen Ding; Omer Khan (HPCA, 2024)

Zero-Space Cost Fault Tolerance for Transformer-based Language Models on ReRAM. - Bingbing Li; Geng Yuan; Zigeng Wang; Shaoyi Huang; Hongwu Peng; Payman Behnam; Wujie Wen; Hang Liu 001; Caiwen Ding (CoRR, 2024)

Enhanced Computationally Efficient Long LoRA Inspired Perceiver Architectures for Auto-Regressive Language Modeling. - Kaleel Mahmood; Shaoyi Huang (CoRR, 2024)

PromptV: Leveraging LLM-powered Multi-Agent Prompting for High-quality Verilog Generation. - Zhendong Mi; Renming Zheng; Haowen Zhong; Yue Sun; Shaoyi Huang (CoRR, 2024)

ASPLOS 2024 Artifact for "MaxK-GNN: Extremely Fast GPU Kernel Design for Accelerating Graph Neural Networks Training". - Hongwu Peng; Xi Xie; Kaustubh Shivdikar; Amit Hasan 001; Jiahui Zhao; Shaoyi Huang; Omer Khan; David R. Kaeli; Caiwen Ding (Zenodo, 2024)

ASPLOS 2024 Artifact for "MaxK-GNN: Extremely Fast GPU Kernel Design for Accelerating Graph Neural Networks Training". - Hongwu Peng; Xi Xie; Kaustubh Shivdikar; Amit Hasan 001; Jiahui Zhao; Shaoyi Huang; Omer Khan; David R. Kaeli; Caiwen Ding (Zenodo, 2024)

KerZOO: Kernel Function Informed Zeroth-Order Optimization for Accurate and Accelerated LLM Fine-Tuning. - Zhendong Mi; Qitao Tan; Xiaodong Yu; Zining Zhu 005; Geng Yuan; Shaoyi Huang (CoRR, 2025)

ACE: Exploring Activation Cosine Similarity and Variance for Accurate and Calibration-Efficient LLM Pruning. - Zhendong Mi; Zhenglun Kong; Geng Yuan; Shaoyi Huang (CoRR, 2025)

Rethinking the Potential of Layer Freezing for Efficient DNN Training. - Chence Yang; Ci Zhang; Le Lu 001; Qitao Tan; Sheng Li; Ao Li 004; Xulong Tang; Shaoyi Huang; Jinzhen Wang; Guoming Li; Jundong Li; Xiaoming Zhai; et al. (CoRR, 2025)