Deeper Machine Translation and Evaluation for German. - Eleftherios Avramidis; Vivien Macketanz; Aljoscha Burchardt; Jindrich Helcl; Hans Uszkoreit (DMTW, 2016)

CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies. - Daniel Zeman; Martin Popel; Milan Straka; Jan Hajic; Joakim Nivre; Filip Ginter; Juhani Luotolahti; Sampo Pyysalo; Slav Petrov; Martin Potthast; Francis M. Tyers; Elena Badmaeva; et al. (CoNLL Shared Task (2), 2017)

Translation Quality and Productivity: A Study on Rich Morphology Languages. - Lucia Specia; Kim Harris; Frédéric Blain; Aljoscha Burchardt; Vivien Macketanz; Inguna Skadina; Matteo Negri; Marco Turchi (MTSummit (1), 2017)

A Linguistic Evaluation of Rule-Based, Phrase-Based, and Neural MT Engines. - Aljoscha Burchardt; Vivien Macketanz; Jon Dehdari; Georg Heigold; Jan-Thorsten Peter; Philip Williams (Prague Bull. Math. Linguistics, 2017)

Fine-grained evaluation of Quality Estimation for Machine translation based on a linguistically motivated Test Suite. - Eleftherios Avramidis; Vivien Macketanz; Arle Lommel; Hans Uszkoreit (PostEditing@AMTA, 2018)

TQ-AutoTest - An Automated Test Suite for (Machine) Translation Quality. - Vivien Macketanz; Renlong Ai; Aljoscha Burchardt; Hans Uszkoreit (LREC, 2018)

Fine-grained evaluation of German-English Machine Translation based on a Test Suite. - Vivien Macketanz; Eleftherios Avramidis; Aljoscha Burchardt; Hans Uszkoreit (WMT (shared task), 2018)

Train, Sort, Explain: Learning to Diagnose Translation Models. - Robert Schwarzenberg; David Harbecke; Vivien Macketanz; Eleftherios Avramidis; Sebastian Möller 001 (NAACL-HLT (Demonstrations), 2019)

Linguistic Evaluation of German-English Machine Translation Using a Test Suite. - Eleftherios Avramidis; Vivien Macketanz; Ursula Strohriegel; Hans Uszkoreit (WMT (2), 2019)

Fine-grained linguistic evaluation for state-of-the-art Machine Translation. - Eleftherios Avramidis; Vivien Macketanz; Ursula Strohriegel; Aljoscha Burchardt; Sebastian Möller 001 (WMT@EMNLP, 2020)

Observing the Learning Curve of NMT Systems With Regard to Linguistic Phenomena. - Patrick Stadler; Vivien Macketanz; Eleftherios Avramidis (ACL (student), 2021)

Linguistic Evaluation for the 2021 State-of-the-art Machine Translation Systems for German to English and English to German. - Vivien Macketanz; Eleftherios Avramidis; Shushen Manakhimova; Sebastian Möller 001 (WMT@EMNLP, 2021)

A new deal for translation quality. - Aljoscha Burchardt; Arle Lommel; Vivien Macketanz (Univers. Access Inf. Soc., 2021)

A Linguistically Motivated Test Suite to Semi-Automatically Evaluate German-English Machine Translation Output. - Vivien Macketanz; Eleftherios Avramidis; Aljoscha Burchardt; He Wang; Renlong Ai; Shushen Manakhimova; Ursula Strohriegel; Sebastian Möller 001; Hans Uszkoreit (LREC, 2022)

A Test Suite for the Evaluation of Portuguese-English Machine Translation. - Mariana Avelino; Vivien Macketanz; Eleftherios Avramidis; Sebastian Möller 001 (PROPOR, 2022)

Linguistically Motivated Evaluation of Machine Translation Metrics Based on a Challenge Set. - Eleftherios Avramidis; Vivien Macketanz (WMT, 2022)

Linguistically Motivated Evaluation of the 2022 State-of-the-art Machine Translation Systems for Three Language Directions. - Vivien Macketanz; Shushen Manakhimova; Eleftherios Avramidis; Ekaterina Lapshinova-Koltunski; Sergei Bagdasarov; Sebastian Möller 001 (WMT, 2022)

Challenging the State-of-the-art Machine Translation Metrics from a Linguistic Perspective. - Eleftherios Avramidis; Shushen Manakhimova; Vivien Macketanz; Sebastian Möller 001 (WMT, 2023)

Linguistically Motivated Evaluation of the 2023 State-of-the-art Machine Translation: Can ChatGPT Outperform NMT? - Shushen Manakhimova; Eleftherios Avramidis; Vivien Macketanz; Ekaterina Lapshinova-Koltunski; Sergei Bagdasarov; Sebastian Möller 001 (WMT, 2023)

Occiglot at WMT24: European Open-source Large Language Models Evaluated on Translation. - Eleftherios Avramidis; Annika Grützner-Zahn; Manuel Brack; Patrick Schramowski; Pedro Ortiz Suarez; Malte Ostendorff; Fabio Barth; Shushen Manakhimova; Vivien Macketanz; Georg Rehm; Kristian Kersting (WMT, 2024)

Machine Translation Metrics Are Better in Evaluating Linguistic Errors on LLMs than on Encoder-Decoder Systems. - Eleftherios Avramidis; Shushen Manakhimova; Vivien Macketanz; Sebastian Möller 001 (WMT, 2024)

Investigating the Linguistic Performance of Large Language Models in Machine Translation. - Shushen Manakhimova; Vivien Macketanz; Eleftherios Avramidis; Ekaterina Lapshinova-Koltunski; Sergei Bagdasarov; Sebastian Möller 001 (WMT, 2024)