MasakhaNER: Named Entity Recognition for African Languages. - David Ifeoluwa Adelani; Jade Z. Abbott; Graham Neubig; Daniel D'souza; Julia Kreutzer; Constantine Lignos; Chester Palen-Michel; Happy Buzaaba; Shruti Rijhwani; Sebastian Ruder; Stephen Mayhew 002; Israel Abebe Azime; et al. (Trans. Assoc. Comput. Linguistics, 2021)

Understanding User Satisfaction with Task-oriented Dialogue Systems. - Clemencia Siro; Mohammad Aliannejadi; Maarten de Rijke (SIGIR, 2022)

Evaluating the Robustness of Machine Reading Comprehension Models to Low Resource Entity Renaming. - Clemencia Siro; Tunde Oluwaseyi Ajayi (AfricaNLP, 2023)

Cross-lingual Open-Retrieval Question Answering for African Languages. - Odunayo Ogundepo; Tajuddeen Gwadabe; Clara Rivera; Jonathan H. Clark; Sebastian Ruder; David Ifeoluwa Adelani; Bonaventure Dossou; Abdou Aziz Diop; Claytone Sikasote; Gilles Hacheme; Happy Buzaaba; Ignatius Ezeani; et al. (EMNLP (Findings), 2023)

MasakhaNEWS: News Topic Classification for African languages. - David Ifeoluwa Adelani; Marek Masiak; Israel Abebe Azime; Jesujoba O. Alabi; Atnafu Lambebo Tonja; Christine Mwase; Odunayo Ogundepo; Bonaventure F. P. Dossou; Akintunde Oladipo; Doreen Nixdorf; Chris Chinenye Emezue; Sana Sabah Al-Azzawi; et al. (IJCNLP (1), 2023)

Evaluating Task-oriented Dialogue Systems with Users. - Clemencia Siro (SIGIR, 2023)

MasakhaNEWS: News Topic Classification for African languages. - David Ifeoluwa Adelani; Marek Masiak; Israel Abebe Azime; Jesujoba Oluwadara Alabi; Atnafu Lambebo Tonja; Christine Mwase; Odunayo Ogundepo; Bonaventure F. P. Dossou; Akintunde Oladipo; Doreen Nixdorf; Chris Chinenye Emezue; Sana Sabah Al-Azzawi; et al. (CoRR, 2023)

AfriQA: Cross-lingual Open-Retrieval Question Answering for African Languages. - Odunayo Ogundepo; Tajuddeen R. Gwadabe; Clara E. Rivera; Jonathan H. Clark; Sebastian Ruder; David Ifeoluwa Adelani; Bonaventure F. P. Dossou; Abdou Aziz Diop; Claytone Sikasote; Gilles Hacheme; Happy Buzaaba; Ignatius Ezeani; et al. (CoRR, 2023)

AfriMTE and AfriCOMET: Empowering COMET to Embrace Under-resourced African Languages. - Jiayi Wang; David Ifeoluwa Adelani; Sweta Agrawal; Ricardo Rei; Eleftheria Briakou; Marine Carpuat; Marek Masiak; Xuanli He; Sofia Bourhim; Andiswa Bukula; Muhidin Mohamed; Temitayo Olatoye; et al. (CoRR, 2023)

Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models. - Aarohi Srivastava; Abhinav Rastogi; Abhishek Rao; Abu Awal Md Shoeb; Abubakar Abid; Adam Fisch; Adam R. Brown; Adam Santoro; Aditya Gupta; Adrià Garriga-Alonso; Agnieszka Kluska; Aitor Lewkowycz; et al. (Trans. Mach. Learn. Res., 2023)

LLMJudge: LLMs for Relevance Judgments. - Hossein A. Rahmani; Emine Yilmaz; Nick Craswell; Bhaskar Mitra 001; Paul Thomas 001; Charles L. A. Clarke; Mohammad Aliannejadi; Clemencia Siro; Guglielmo Faggioli (LLM4Eval@SIGIR, 2024)

Context Does Matter: Implications for Crowdsourced Evaluation Labels in Task-Oriented Dialogue Systems. - Clemencia Siro; Mohammad Aliannejadi; Maarten de Rijke (NAACL-HLT (Findings), 2024)

AfriMTE and AfriCOMET: Enhancing COMET to Embrace Under-resourced African Languages. - Jiayi Wang; David Ifeoluwa Adelani; Sweta Agrawal; Marek Masiak; Ricardo Rei; Eleftheria Briakou; Marine Carpuat; Xuanli He; Sofia Bourhim; Andiswa Bukula; Muhidin Mohamed; Temitayo Olatoye; et al. (NAACL-HLT, 2024)

Rethinking the Evaluation of Dialogue Systems: Effects of User Feedback on Crowdworkers and LLMs. - Clemencia Siro; Mohammad Aliannejadi; Maarten de Rijke (SIGIR, 2024)

Asking Multimodal Clarifying Questions in Mixed-Initiative Conversational Search. - Yifei Yuan 002; Clemencia Siro; Mohammad Aliannejadi; Maarten de Rijke; Wai Lam (WWW, 2024)

AGENT-CQ: Automatic Generation and Evaluation of Clarifying Questions for Conversational Search with LLMs. - Clemencia Siro; Yifei Yuan 002; Mohammad Aliannejadi; Maarten de Rijke (CoRR, 2024)

Understanding and Predicting User Satisfaction with Conversational Recommender Systems. - Clemencia Siro; Mohammad Aliannejadi; Maarten de Rijke (ACM Trans. Inf. Syst., 2024)

Report on the 1st Workshop on Large Language Model for Evaluation in Information Retrieval (LLM4Eval 2024) at SIGIR 2024. - Hossein A. Rahmani; Clemencia Siro; Mohammad Aliannejadi; Nick Craswell; Charles L. A. Clarke; Guglielmo Faggioli; Bhaskar Mitra 001; Paul Thomas 001; Emine Yilmaz (SIGIR Forum, 2024)

Do Images Clarify? A Study on the Effect of Images on Clarifying Questions in Conversational Search. - Clemencia Siro; Zahra Abbasiantaeb; Yifei Yuan 002; Mohammad Aliannejadi; Maarten de Rijke (CHIIR, 2025)

LLM4Eval: Large Language Model for Evaluation in IR. - Clemencia Siro; Hossein A. Rahmani; Mohammad Aliannejadi; Nick Craswell; Charles L. A. Clarke; Guglielmo Faggioli; Bhaskar Mitra 001; Paul Thomas 001; Emine Yilmaz (SIGIR, 2025)

LLM4Eval@WSDM 2025: Large Language Model for Evaluation in Information Retrieval. - Hossein A. Rahmani; Clemencia Siro; Mohammad Aliannejadi; Nick Craswell; Charles L. A. Clarke; Guglielmo Faggioli; Bhaskar Mitra 001; Paul Thomas 001; Emine Yilmaz (WSDM, 2025)

Multi-Turn Multi-Modal Question Clarification for Enhanced Conversational Understanding. - Kimia Ramezan; Alireza Amiri Bavandpour; Yifei Yuan 002; Clemencia Siro; Mohammad Aliannejadi (CoRR, 2025)

Judging the Judges: A Collection of LLM-Generated Relevance Judgements. - Hossein A. Rahmani; Clemencia Siro; Mohammad Aliannejadi; Nick Craswell; Charles L. A. Clarke; Guglielmo Faggioli; Bhaskar Mitra 001; Paul Thomas 001; Emine Yilmaz (CoRR, 2025)