Linguistic Evaluation for the 2021 State-of-the-art Machine Translation Systems for German to English and English to German. - Vivien Macketanz; Eleftherios Avramidis; Shushen Manakhimova; Sebastian Möller 001 (WMT@EMNLP, 2021)

A Linguistically Motivated Test Suite to Semi-Automatically Evaluate German-English Machine Translation Output. - Vivien Macketanz; Eleftherios Avramidis; Aljoscha Burchardt; He Wang; Renlong Ai; Shushen Manakhimova; Ursula Strohriegel; Sebastian Möller 001; Hans Uszkoreit (LREC, 2022)

Linguistically Motivated Evaluation of the 2022 State-of-the-art Machine Translation Systems for Three Language Directions. - Vivien Macketanz; Shushen Manakhimova; Eleftherios Avramidis; Ekaterina Lapshinova-Koltunski; Sergei Bagdasarov; Sebastian Möller 001 (WMT, 2022)

Challenging the State-of-the-art Machine Translation Metrics from a Linguistic Perspective. - Eleftherios Avramidis; Shushen Manakhimova; Vivien Macketanz; Sebastian Möller 001 (WMT, 2023)

Linguistically Motivated Evaluation of the 2023 State-of-the-art Machine Translation: Can ChatGPT Outperform NMT? - Shushen Manakhimova; Eleftherios Avramidis; Vivien Macketanz; Ekaterina Lapshinova-Koltunski; Sergei Bagdasarov; Sebastian Möller 001 (WMT, 2023)

Occiglot at WMT24: European Open-source Large Language Models Evaluated on Translation. - Eleftherios Avramidis; Annika Grützner-Zahn; Manuel Brack; Patrick Schramowski; Pedro Ortiz Suarez; Malte Ostendorff; Fabio Barth; Shushen Manakhimova; Vivien Macketanz; Georg Rehm; Kristian Kersting (WMT, 2024)

Machine Translation Metrics Are Better in Evaluating Linguistic Errors on LLMs than on Encoder-Decoder Systems. - Eleftherios Avramidis; Shushen Manakhimova; Vivien Macketanz; Sebastian Möller 001 (WMT, 2024)

Investigating the Linguistic Performance of Large Language Models in Machine Translation. - Shushen Manakhimova; Vivien Macketanz; Eleftherios Avramidis; Ekaterina Lapshinova-Koltunski; Sergei Bagdasarov; Sebastian Möller 001 (WMT, 2024)