How does the pre-training objective affect what large language models learn about linguistic properties? - Ahmed Alajrami; Nikolaos Aletras (ACL (2), 2022)

Understanding the Role of Input Token Characters in Language Models: How Does Information Loss Affect Performance? - Ahmed Alajrami; Katerina Margatina; Nikolaos Aletras (EMNLP, 2023)