Understanding and Improving Knowledge Distillation for Quantization Aware Training of Large Transformer Encoders. - Minsoo Kim; Sihwa Lee; Sukjin Hong; Du-Seong Chang; Jungwook Choi (EMNLP, 2022)

Revisiting Intermediate Layer Distillation for Compressing Language Models: An Overfitting Perspective. - Jongwoo Ko; Seungjoon Park; Minchan Jeong; Sukjin Hong; Euijai Ahn; Du-Seong Chang; Se-Young Yun (EACL (Findings), 2023)

Token-Scaled Logit Distillation for Ternary Weight Generative Language Models. - Minsoo Kim; Sihwa Lee; Janghwan Lee; Sukjin Hong; Du-Seong Chang; Wonyong Sung; Jungwook Choi (NeurIPS, 2023)

Improving Conversational Abilities of Quantized Large Language Models via Direct Preference Alignment. - Janghwan Lee; Seongmin Park 003; Sukjin Hong; Minsoo Kim; Du-Seong Chang; Jungwook Choi (ACL (1), 2024)

RILQ: Rank-Insensitive LoRA-Based Quantization Error Compensation for Boosting 2-Bit Large Language Model Accuracy. - Geonho Lee; Janghwan Lee; Sukjin Hong; Minsoo Kim; Euijai Ahn; Du-Seong Chang; Jungwook Choi (AAAI, 2025)