Explore Better Relative Position Embeddings from Encoding Perspective for Transformer Models. - Anlin Qu; Jianwei Niu 002; Shasha Mo (EMNLP (1), 2021)

Enhancing Transformer with Horizontal and Vertical Guiding Mechanisms for Neural Language Modeling. - Anlin Qu; Jianwei Niu 002; Shasha Mo (ICC, 2021)