Application of Attention Mechanism with Prior Information in Natural Language Processing. - Lingling Zhang; Zhenxiong Zhou; Pengyu Ji; Aoxue Mei (Int. J. Artif. Intell. Tools, 2022)

Improving Span Representation by Efficient Span-Level Attention. - Pengyu Ji; Songlin Yang; Kewei Tu (EMNLP (Findings), 2023)

Generative Pretrained Structured Transformers: Unsupervised Syntactic Language Models at Scale. - Xiang Hu; Pengyu Ji; Qingyang Zhu; Wei Wu; Kewei Tu (ACL (1), 2024)

Unsupervised Morphological Tree Tokenizer. - Qingyang Zhu; Xiang Hu; Pengyu Ji; Wei Wu; Kewei Tu (ACL (Findings), 2025)