Ian Soboroff

Proceedings of the MM '22: The 30th ACM International Conference on Multimedia, Lisboa, Portugal, October 10, 2022

Incident Streams 2021 Off the Deep End: Deeper Annotations and Evaluations in Twitter.

[BibT_eX]

[DOI]

Proceedings of the 19th International Conference on Information Systems for Crisis Response and Management, 2022

Second International Workshop on Deep Video Understanding.

[BibT_eX]

[DOI]

Proceedings of the International Conference on Multimodal Interaction, 2022

CHIIR Workshop on Audio Collection Human Interaction (AudioCHI 2022): http: //speechretrievalworkshop.github.io.

[BibT_eX]

[DOI]

Proceedings of the CHIIR '22: ACM SIGIR Conference on Human Information Interaction and Retrieval, Regensburg, Germany, March 14, 2022

2021

Introduction to the second issue on machine translation for low-resource languages.

[BibT_eX]

[DOI]

Mach. Transl., 2021

Searching for scientific evidence in a pandemic: An overview of TREC-COVID.

[BibT_eX]

[DOI]

J. Biomed. Informatics, 2021

Podcast Metadata and Content: Episode Relevance andAttractiveness in Ad Hoc Search.

[BibT_eX]

[DOI]

CoRR, 2021

Overview of TREC 2021.

[BibT_eX]

[DOI]

Proceedings of the Thirtieth Text REtrieval Conference, 2021

TREC Deep Learning Track: Reusable Test Collections in the Large Data Regime.

[BibT_eX]

[DOI]

Proceedings of the SIGIR '21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2021

Podcast Metadata and Content: Episode Relevance and Attractiveness in Ad Hoc Search.

[BibT_eX]

[DOI]

Proceedings of the SIGIR '21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2021

2020

TREC-COVID: constructing a pandemic information retrieval test collection.

[BibT_eX]

[DOI]

SIGIR Forum, 2020

Introduction to the Special Issue on Machine Translation for Low-Resource Languages.

[BibT_eX]

[DOI]

Mach. Transl., 2020

TREC-COVID: rationale and structure of an information retrieval shared task for COVID-19.

[BibT_eX]

[DOI]

J. Am. Medical Informatics Assoc., 2020

Evaluating Multimedia and Language Tasks.

[BibT_eX]

[DOI]

Frontiers Artif. Intell., 2020

TREC 2020 News Track Overview.

[BibT_eX]

[DOI]

Shudong Huang

Proceedings of the Twenty-Ninth Text REtrieval Conference, 2020

Incident Streams 2020: TRECIS in the Time of COVID-19.

[BibT_eX]

[DOI]

Proceedings of the Twenty-Ninth Text REtrieval Conference, 2020

How to Measure the Reproducibility of System-oriented IR Experiments.

[BibT_eX]

[DOI]

Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, 2020

HLVU: A New Challenge to Test Deep Understanding of Movies the Way Humans do.

[BibT_eX]

[DOI]

Proceedings of the 2020 on International Conference on Multimedia Retrieval, 2020

Incident Streams 2019: Actionable Insights and How to Find Them.

[BibT_eX]

[DOI]

Proceedings of the 17th International Conference on Information Systems for Crisis Response and Management, 2020

International Workshop on Deep Video Understanding.

[BibT_eX]

[DOI]

Proceedings of the ICMI '20: International Conference on Multimodal Interaction, 2020

2019

On enhancing the robustness of timeline summarization test collections.

[BibT_eX]

[DOI]

Inf. Process. Manag., 2019

Overview of TAC-KBP 2019 Fine-grained Entity Extraction.

[BibT_eX]

[DOI]

Proceedings of the 2019 Text Analysis Conference, 2019

Financial Entity Identification and Information Integration (FEIII) 2019 Challenge: The Report of the Organizing Committee.

[BibT_eX]

[DOI]

Proceedings of the 5th Workshop on Data Science for Macro-modeling with Financial and Economic Datasets, 2019

TREC Incident Streams: Finding Actionable Information on Social Media.

[BibT_eX]

[DOI]

Proceedings of the 16th International Conference on Information Systems for Crisis Response and Management, 2019

EVIA 2019 Preface.

[BibT_eX]

[DOI]

Nicola Ferro

Min Zhang

Proceedings of the 9th International Workshop on Evaluating Information Access co-located with the 14th NTCIR Conference on the Evaluation of Information Access Technologies (NTCIR 2019), 2019

CENTRE@CLEF 2019.

[BibT_eX]

[DOI]

Proceedings of the Advances in Information Retrieval, 2019

Overview of CENTRE@CLEF 2019: Sequel in the Systematic Reproducibility Realm.

[BibT_eX]

[DOI]

Proceedings of the Experimental IR Meets Multilinguality, Multimodality, and Interaction, 2019

CENTRE@CLEF2019: Overview of the Replicability and Reproducibility Tasks.

[BibT_eX]

[DOI]

Proceedings of the Working Notes of CLEF 2019, 2019

2018

Report on GLARE 2018: 1st Workshop on Generalization in Information Retrieval.

[BibT_eX]

[DOI]

Nicola Ferro

Norbert Fuhr

SIGIR Forum, 2018

Report on EVIA 2017.

[BibT_eX]

[DOI]

Nicola Ferro

SIGIR Forum, 2018

The Dagstuhl Perspectives Workshop on Performance Modeling and Prediction.

[BibT_eX]

[DOI]

SIGIR Forum, 2018

Overview of the NIST 2016 LoReHLT evaluation.

[BibT_eX]

[DOI]

Mach. Transl., 2018

Preface.

[BibT_eX]

[DOI]

Audrey Tong

Mach. Transl., 2018

From Evaluating to Forecasting Performance: How to Turn Information Retrieval, Natural Language Processing and Recommender Systems into Predictive Sciences (Dagstuhl Perspectives Workshop 17442).

[BibT_eX]

[DOI]

Dagstuhl Manifestos, 2018

TREC 2018 News Track Overview.

[BibT_eX]

[DOI]

Shudong Huang

Proceedings of the Twenty-Seventh Text REtrieval Conference, 2018

Financial Entity Identification and Information Integration (FEIII) 2018 Challenge: The Report of the Organizing Committee.

[BibT_eX]

[DOI]

Proceedings of the Fourth International Workshop on Data Science for Macro-Modeling with Financial and Economic Datasets, 2018

TREC 2018 News Track.

[BibT_eX]

[DOI]

Shudong Huang

Proceedings of the Second International Workshop on Recent Trends in News Information Retrieval co-located with 40th European Conference on Information Retrieval (ECIR 2018), 2018

Overview of CENTRE@CLEF 2018: A First Tale in the Systematic Reproducibility Realm.

[BibT_eX]

[DOI]

Proceedings of the Experimental IR Meets Multilinguality, Multimodality, and Interaction, 2018

CENTRE@CLEF2018: Overview of the Replicability Task.

[BibT_eX]

[DOI]

Proceedings of the Working Notes of CLEF 2018, 2018

Meta-Analysis for Retrieval Experiments Involving Multiple Test Collections.

[BibT_eX]

[DOI]

Proceedings of the 27th ACM International Conference on Information and Knowledge Management, 2018

2017

Using Replicates in Information Retrieval Evaluation.

[BibT_eX]

[DOI]

Daniel Samarov

ACM Trans. Inf. Syst., 2017

TREC 2017 Dynamic Domain Track Overview.

[BibT_eX]

[DOI]

Grace Hui Yang

Zhiwen Tang

Proceedings of The Twenty-Sixth Text REtrieval Conference, 2017

Financial Entity Identification and Information Integration (FEIII) 2017 Challenge: The Report of the Organizing Committee.

[BibT_eX]

[DOI]

Proceedings of the 3rd International Workshop on Data Science for Macro-Modeling with Financial and Economic Datasets, 2017

Building Test Collections: An Interactive Guide for Students and Others Without Their Own Evaluation Conference Series.

[BibT_eX]

[DOI]

Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2017

New Tracks for TREC 2018.

[BibT_eX]

[DOI]

Proceedings of the 13th NTCIR Conference, 2017

2016

TREC 2016 Dynamic Domain Track Overview.

[BibT_eX]

[DOI]

Grace Hui Yang

Proceedings of The Twenty-Fifth Text REtrieval Conference, 2016

Privacy-Preserving IR 2016: Differential Privacy, Search, and Social Media.

[BibT_eX]

[DOI]

Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval, 2016

The BOLT IR Test Collections of Multilingual Passage Retrieval from Discussion Forums.

[BibT_eX]

[DOI]

Kira Griffitt

Stephanie M. Strassel

Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval, 2016

Promoting Repeatability Through Open Runs.

[BibT_eX]

[DOI]

Shahzad Rajput

Proceedings of the Seventh International Workshop on Evaluating Information Access, 2016

News from TREC.

[BibT_eX]

[DOI]

Proceedings of the 12th NTCIR Conference on Evaluation of Information Access Technologies, 2016

Financial Entity Identification and Information Integration (FEIII) Challenge: The Report of the Organizing Committee.

[BibT_eX]

[DOI]

Proceedings of the Second International Workshop on Data Science for Macro-Modeling, 2016

2015

Privacy Preserving IR 2015: A SIGIR 2015 Workshop.

[BibT_eX]

[DOI]

Grace Hui Yang

SIGIR Forum, 2015

TREC 2015 Dynamic Domain Track Overview.

[BibT_eX]

[DOI]

Hui Yang

John R. Frank

Proceedings of The Twenty-Fourth Text REtrieval Conference, 2015

Privacy-Preserving IR 2015: When Information Retrieval Meets Privacy and Security.

[BibT_eX]

[DOI]

Hui Yang

Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2015

2014

Building Better Search Engines by Measuring Search Quality.

[BibT_eX]

[DOI]

Paul Over

IT Prof., 2014

Evaluating Stream Filtering for Entity Profile Updates in TREC 2012, 2013, and 2014.

[BibT_eX]

[DOI]

Proceedings of The Twenty-Third Text REtrieval Conference, 2014

Computing Confidence Intervals for Common IR Measures.

[BibT_eX]

[DOI]

Proceedings of the Sixth International Workshop on Evaluating Information Access, 2014

2013

Introduction to the special issue on search intents and diversification.

[BibT_eX]

[DOI]

Inf. Retr., 2013

Evaluating Stream Filtering for Entity Profile Updates for TREC 2013.

[BibT_eX]

[DOI]

Proceedings of The Twenty-Second Text REtrieval Conference, 2013

Building test collections: an interactive tutorial for students and others without their own evaluation conference series.

[BibT_eX]

[DOI]

Proceedings of the 36th International ACM SIGIR conference on research and development in Information Retrieval, 2013

2012

Information Retrieval on the Blogosphere.

[BibT_eX]

[DOI]

Found. Trends Inf. Retr., 2012

Overview of the TREC-2012 Microblog Track.

[BibT_eX]

[DOI]

Proceedings of The Twenty-First Text REtrieval Conference, 2012

Building an Entity-Centric Stream Filtering Test Collection for TREC 2012.

[BibT_eX]

[DOI]

Proceedings of The Twenty-First Text REtrieval Conference, 2012

On building a reusable Twitter corpus.

[BibT_eX]

[DOI]

Proceedings of the 35th International ACM SIGIR conference on research and development in Information Retrieval, 2012

2011

A comparative analysis of cascade measures for novelty and diversity.

[BibT_eX]

[DOI]

Proceedings of the Forth International Conference on Web Search and Web Data Mining, 2011

Overview of the TREC 2011 Microblog Track.

[BibT_eX]

[DOI]

Proceedings of The Twentieth Text REtrieval Conference, 2011

Overview of the TREC 2011 Web Track.

[BibT_eX]

[DOI]

Proceedings of The Twentieth Text REtrieval Conference, 2011

Reports from Other Evaluation Campaigns and Panel Discussion: TREC is 20 Years Old, Where Now for Evaluation Campaigns?

[BibT_eX]

[DOI]

Proceedings of the 4th International Workshop on Evaluating Information Access, 2011

2010

Blog track research at TREC.

[BibT_eX]

[DOI]

SIGIR Forum, 2010

Overview of the TREC 2010 Blog Track.

[BibT_eX]

[DOI]

Proceedings of The Nineteenth Text REtrieval Conference, 2010

Overview of the TREC 2010 Web Track.

[BibT_eX]

[DOI]

Proceedings of The Nineteenth Text REtrieval Conference, 2010

The effect of assessor error on IR system evaluation.

[BibT_eX]

[DOI]

Ben Carterette

Proceedings of the Proceeding of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval, 2010

Test Collection Diagnosis and Treatment.

[BibT_eX]

[DOI]

Proceedings of the 3rd International Workshop on Evaluating Information Access, 2010

2009

The search and social media workshop at SIGIR 2009.

[BibT_eX]

[DOI]

Eugene Agichtein

Marti A. Hearst

SIGIR Forum, 2009

A guide to the RIA workshop data archive.

[BibT_eX]

[DOI]

Inf. Retr., 2009

Overview of the TREC 2009 Blog Track.

[BibT_eX]

[DOI]

Proceedings of The Eighteenth Text REtrieval Conference, 2009

Overview of the TREC 2009 Web Track.

[BibT_eX]

[DOI]

Proceedings of The Eighteenth Text REtrieval Conference, 2009

Is spam an issue for opinionated blog post search?

[BibT_eX]

[DOI]

Proceedings of the 32nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2009

2008

Overview of the TREC 2008 Blog Track.

[BibT_eX]

[DOI]

Craig MacDonald

Proceedings of The Seventeenth Text REtrieval Conference, 2008

Overview of the TREC 2008 Enterprise Track.

[BibT_eX]

[DOI]

Proceedings of The Seventeenth Text REtrieval Conference, 2008

Limits of opinion-finding baseline systems.

[BibT_eX]

[DOI]

Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2008

Relevance assessment: are judges exchangeable and does it matter.

[BibT_eX]

[DOI]

Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2008

TREC, TAC, takeoffs, tacks, tasks, and titillations for 2009.

[BibT_eX]

[DOI]

Proceedings of the 7th NTCIR Workshop Meeting on Evaluation of Information Access Technologies: Information Retrieval, 2008

On the TREC Blog Track.

[BibT_eX]

[DOI]

Proceedings of the Second International Conference on Weblogs and Social Media, 2008

2007

The CSIRO enterprise search test collection.

[BibT_eX]

[DOI]

SIGIR Forum, 2007

Bias and the limits of pooling for large collections.

[BibT_eX]

[DOI]

Inf. Retr., 2007

Overview of the TREC 2007 Blog Track.

[BibT_eX]

[DOI]

Proceedings of The Sixteenth Text REtrieval Conference, 2007

Overview of the TREC 2007 Enterprise Track.

[BibT_eX]

[DOI]

Proceedings of The Sixteenth Text REtrieval Conference, 2007

A comparison of pooled and sampled relevance judgments.

[BibT_eX]

[DOI]

Proceedings of the SIGIR 2007: Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2007

Problems with Kendall's tau.

[BibT_eX]

[DOI]

Mark Sanderson

Proceedings of the SIGIR 2007: Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2007

Reliable information retrieval evaluation with incomplete and biased judgements.

[BibT_eX]

[DOI]

Proceedings of the SIGIR 2007: Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2007

A Comparison of Pooled and Sampled Relevance Judgments in the TREC 2006 Terabyte Track.

[BibT_eX]

[DOI]

Proceedings of the 1st International Workshop on Evaluating Information Access, 2007

What's New at TREC: Blog and Legal Discovery Search at TREC-2006.

[BibT_eX]

[DOI]

Proceedings of the 6th NTCIR Workshop Meeting on Evaluation of Information Access Technologies: Information Retrieval, 2007

2006

Overview of the TREC 2006 Enterprise Track.

[BibT_eX]

[DOI]

Arjen P. de Vries

Proceedings of the Fifteenth Text REtrieval Conference, 2006

Overview of the TREC 2006 Blog Track.

[BibT_eX]

[DOI]

Proceedings of the Fifteenth Text REtrieval Conference, 2006

The TREC 2006 Terabyte Track.

[BibT_eX]

[DOI]

Stefan Büttcher

Proceedings of the Fifteenth Text REtrieval Conference, 2006

Dynamic test collections: measuring search effectiveness on the live web.

[BibT_eX]

[DOI]

Proceedings of the SIGIR 2006: Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2006

Bias and the limits of pooling.

[BibT_eX]

[DOI]

Proceedings of the SIGIR 2006: Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2006

An Exploratory Study of the W3C Mailing List Test Collection for Retrieval of Emails with Pro/Con Argument.

[BibT_eX]

[DOI]

Yejun Wu

Douglas W. Oard

Proceedings of the CEAS 2006, 2006

2005

The TREC terabyte retrieval track.

[BibT_eX]

[DOI]

SIGIR Forum, 2005

SIGIR workshop report: predicting query difficulty - methods and applications.

[BibT_eX]

[DOI]

David Carmel

Elad Yom-Tov

SIGIR Forum, 2005

Overview of the TREC 2005 Enterprise Track.

[BibT_eX]

[DOI]

Arjen P. de Vries

Proceedings of the Fourteenth Text REtrieval Conference, 2005

The TREC 2005 Terabyte Track.

[BibT_eX]

[DOI]

Falk Scholer

Proceedings of the Fourteenth Text REtrieval Conference, 2005

Novelty Detection: The TREC Experience.

[BibT_eX]

[DOI]

Proceedings of the HLT/EMNLP 2005, 2005

2004

Overview of the TREC 2004 Novelty Track.

[BibT_eX]

[DOI]

Proceedings of the Thirteenth Text REtrieval Conference, 2004

Overview of the TREC 2004 Terabyte Track.

[BibT_eX]

[DOI]

Proceedings of the Thirteenth Text REtrieval Conference, 2004

On evaluating web search with very few relevant documents.

[BibT_eX]

[DOI]

Proceedings of the SIGIR 2004: Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2004

2003

Summary of the SIGIR 2003 workshop on defining evaluation methodologies for terabyte-scale test collections.

[BibT_eX]

[DOI]

SIGIR Forum, 2003

Overview of the TREC 2003 Novelty Track.

[BibT_eX]

[DOI]

Proceedings of The Twelfth Text REtrieval Conference, 2003

Building a filtering test collection for TREC 2002.

[BibT_eX]

[DOI]

Stephen E. Robertson

Proceedings of the SIGIR 2003: Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, July 28, 2003

2002

Do TREC web collections look like the web?

[BibT_eX]

[DOI]

SIGIR Forum, 2002

Related, but not Relevant: Content-Based Collaborative Filtering in TREC-8.

[BibT_eX]

[DOI]

Inf. Retr., 2002

ITtalks: A Case Study in the Semantic Web and DAML+OIL.

[BibT_eX]

[DOI]

IEEE Intell. Syst., 2002

The TREC 2002 Filtering Track Report.

[BibT_eX]

[DOI]

Stephen E. Robertson

Proceedings of The Eleventh Text REtrieval Conference, 2002

Does WT10g look like the web?.

[BibT_eX]

[DOI]

Proceedings of the SIGIR 2002: Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2002

Automatic evaluation of world wide web search services.

[BibT_eX]

[DOI]

Abdur Chowdhury

Proceedings of the SIGIR 2002: Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2002

2001

The TREC 2001 Filtering Track Report.

[BibT_eX]

[DOI]

Stephen E. Robertson

Proceedings of The Tenth Text REtrieval Conference, 2001

Ranking Retrieval Systems without Relevance Judgments.

[BibT_eX]

[DOI]

Patrick Cahan

Proceedings of the SIGIR 2001: Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2001

2000

Collaborative filtering and the generalized vector space model.

[BibT_eX]

[DOI]

Proceedings of the SIGIR 2000: Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2000

1999

Workshop on Recommender Systems: Algorithms and Evaluation.

[BibT_eX]

[DOI]