A Study on Knowledge Distillation from Weak Teacher for Scaling Up Pre-trained Language Models. - Hayeon Lee; Rui Hou 007; Jongpil Kim; Davis Liang; Sung Ju Hwang; Alexander Min (ACL (Findings), 2023)

Co-training and Co-distillation for Quality Improvement and Compression of Language Models. - Hayeon Lee; Rui Hou 007; Jongpil Kim; Davis Liang; Hongbo Zhang; Sung Ju Hwang; Alexander Min (EMNLP (Findings), 2023)