Ellen M. Voorhees

Orcid: 0000-0002-5658-2308

Affiliations:
  • National Institute of Standards and Technology (NIST), Gaithersburg, MD, USA


According to our database1, Ellen M. Voorhees authored at least 136 papers between 1984 and 2022.

Collaborative distances:
  • Dijkstra number2 of four.
  • Erdős number3 of four.

Awards

ACM Fellow

ACM Fellow 2018, "For contributions in evaluation of information retrieval, question answering, and other language technologies".

Timeline

Legend:

Book 
In proceedings 
Article 
PhD thesis 
Dataset
Other 

Links

Online presence:

On csauthors.net:

Bibliography

2022
Introduction to the Special Issue on Computational Methods for Biomedical NLP.
ACM Trans. Comput. Heal., 2022

Can Old TREC Collections Reliably Evaluate Modern Neural Retrieval Models?
CoRR, 2022

Overview of the TREC 2022 Clinical Trials Track.
Proceedings of the Thirty-First Text REtrieval Conference, 2022

Overview of the TREC 2022 Deep Learning Track.
Proceedings of the Thirty-First Text REtrieval Conference, 2022

Human Preferences as Dueling Bandits.
Proceedings of the SIGIR '22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Madrid, Spain, July 11, 2022

Too Many Relevants: Whither Cranfield Test Collections?
Proceedings of the SIGIR '22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Madrid, Spain, July 11, 2022

2021
Searching for scientific evidence in a pandemic: An overview of TREC-COVID.
J. Biomed. Informatics, 2021

On the Quality of the TREC-COVID IR Test Collections.
Proceedings of the SIGIR '21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2021

TREC Deep Learning Track: Reusable Test Collections in the Large Data Regime.
Proceedings of the SIGIR '21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2021

2020
TREC-COVID: constructing a pandemic information retrieval test collection.
SIGIR Forum, 2020

Coopetition in IR research.
SIGIR Forum, 2020

TREC-COVID: rationale and structure of an information retrieval shared task for COVID-19.
J. Am. Medical Informatics Assoc., 2020

Overview of the TREC 2019 deep learning track.
CoRR, 2020

Overview of the TREC 2020 Precision Medicine Track.
Proceedings of the Twenty-Ninth Text REtrieval Conference, 2020

2019
Overview of the TREC 2019 Precision Medicine Track.
Proceedings of the Twenty-Eighth Text REtrieval Conference, 2019

The Evolution of Cranfield.
Proceedings of the Information Retrieval Evaluation in a Changing World, 2019

2018
Overview of the TREC 2018 Precision Medicine Track.
Proceedings of the Twenty-Seventh Text REtrieval Conference, 2018

On Building Fair and Reusable Test Collections using Bandit Techniques.
Proceedings of the 27th ACM International Conference on Information and Knowledge Management, 2018

Benchmarking Information Retrieval for Precision Oncology: the TREC Precision Medicine Track.
Proceedings of the AMIA 2018, 2018

2017
Using Replicates in Information Retrieval Evaluation.
ACM Trans. Inf. Syst., 2017

Evaluating Evaluation Measure Stability.
SIGIR Forum, 2017

Overview of the TREC 2017 Precision Medicine Track.
Proceedings of The Twenty-Sixth Text REtrieval Conference, 2017

Overview of the TREC 2017 Real-Time Summarization Track.
Proceedings of The Twenty-Sixth Text REtrieval Conference, 2017

TREC 2017 Common Core Track Overview.
Proceedings of The Twenty-Sixth Text REtrieval Conference, 2017

2016
Women in IR.
SIGIR Forum, 2016

State-of-the-art in biomedical literature retrieval for clinical cases: a survey of the TREC 2014 CDS track.
Inf. Retr. J., 2016

Overview of the TREC 2016 Clinical Decision Support Track.
Proceedings of The Twenty-Fifth Text REtrieval Conference, 2016

Overview of the TREC 2016 Real-Time Summarization Track.
Proceedings of The Twenty-Fifth Text REtrieval Conference, 2016

Overview of the TREC 2016 Contextual Suggestion Track.
Proceedings of The Twenty-Fifth Text REtrieval Conference, 2016

Promoting Repeatability Through Open Runs.
Proceedings of the Seventh International Workshop on Evaluating Information Access, 2016

2015
Overview of the TREC 2015 Clinical Decision Support Track.
Proceedings of The Twenty-Fourth Text REtrieval Conference, 2015

Overview of the TREC-2015 Microblog Track.
Proceedings of The Twenty-Fourth Text REtrieval Conference, 2015

Overview of the TREC 2015 Contextual Suggestion Track.
Proceedings of The Twenty-Fourth Text REtrieval Conference, 2015

On the Behavior of PRES Using Incomplete Judgment Sets.
Proceedings of the 2015 International Conference on The Theory of Information Retrieval, 2015

2014
Building Better Search Engines by Measuring Search Quality.
IT Prof., 2014

Overview of the TREC 2014 Clinical Decision Support Track.
Proceedings of The Twenty-Third Text REtrieval Conference, 2014

Evaluating Stream Filtering for Entity Profile Updates in TREC 2012, 2013, and 2014.
Proceedings of The Twenty-Third Text REtrieval Conference, 2014

Overview of the TREC 2014 Contextual Suggestion Track.
Proceedings of The Twenty-Third Text REtrieval Conference, 2014

TREC 2014 Web Track Overview.
Proceedings of The Twenty-Third Text REtrieval Conference, 2014

On run diversity in Evaluation as a Service.
Proceedings of the 37th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2014

The effect of sampling strategy on inferred measures.
Proceedings of the 37th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2014

2013
Evaluating Stream Filtering for Entity Profile Updates for TREC 2013.
Proceedings of The Twenty-Second Text REtrieval Conference, 2013

Overview of the TREC 2013 Contextual Suggestion Track.
Proceedings of The Twenty-Second Text REtrieval Conference, 2013

TREC 2013 Web Track Overview.
Proceedings of The Twenty-Second Text REtrieval Conference, 2013

The TREC Medical Records Track.
Proceedings of the ACM Conference on Bioinformatics, 2013

2012
Overview of the TREC 2012 Medical Records Track.
Proceedings of The Twenty-First Text REtrieval Conference, 2012

Overview of the TREC 2012 Contextual Suggestion Track.
Proceedings of The Twenty-First Text REtrieval Conference, 2012

Overview of the TREC 2012 Web Track.
Proceedings of The Twenty-First Text REtrieval Conference, 2012

2011
Overview of the TREC 2011 Web Track.
Proceedings of The Twentieth Text REtrieval Conference, 2011

Overview of Information Retrieval Evaluation.
Proceedings of the Current Challenges in Patent Information Retrieval, 2011

2010
Reflections on TREC QA.
Proceedings of the CLEF 2010 LABs and Workshops, 2010

2009
Report on the SIGIR 2009 workshop on the future of IR evaluation.
SIGIR Forum, 2009

On the number of terms used in automatic query expansion.
Inf. Retr., 2009

TREC genomics special issue overview.
Inf. Retr., 2009

Topic set size redux.
Proceedings of the 32nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2009

2008
On test collections for adaptive information retrieval.
Inf. Process. Manag., 2008

Contradictions and Justifications: Extensions to the Textual Entailment Task.
Proceedings of the ACL 2008, 2008

2007
Bias and the limits of pooling for large collections.
Inf. Retr., 2007

TREC: Continuing information retrieval's tradition of experimentation.
Commun. ACM, 2007

Overview of TREC 2007.
Proceedings of The Sixteenth Text REtrieval Conference, 2007

2006
The TREC 2005 robust track.
SIGIR Forum, 2006

TREC: An overview.
Annu. Rev. Inf. Sci. Technol., 2006

Overview of the TREC 2006.
Proceedings of the Fifteenth Text REtrieval Conference, 2006

Bias and the limits of pooling.
Proceedings of the SIGIR 2006: Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2006

A complex document information processing prototype.
Proceedings of the SIGIR 2006: Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2006

2005
The TREC robust retrieval track.
SIGIR Forum, 2005

Overview of the TREC 2005 Question Answering Track.
Proceedings of the Fourteenth Text REtrieval Conference, 2005

Overview of the TREC 2005 Robust Retrieval Track.
Proceedings of the Fourteenth Text REtrieval Conference, 2005

Overview of TREC 2005.
Proceedings of the Fourteenth Text REtrieval Conference, 2005

Using Question Series to Evaluate Question Answering System Effectiveness.
Proceedings of the HLT/EMNLP 2005, 2005

2004
Overview of the TREC 2004 Robust Track.
Proceedings of the Thirteenth Text REtrieval Conference, 2004

Overview of the TREC 2004 Question Answering Track.
Proceedings of the Thirteenth Text REtrieval Conference, 2004

Overview of TREC 2004.
Proceedings of the Thirteenth Text REtrieval Conference, 2004

Measuring ineffectiveness.
Proceedings of the SIGIR 2004: Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2004

Retrieval evaluation with incomplete information.
Proceedings of the SIGIR 2004: Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2004

2003
Summary of the SIGIR 2003 workshop on defining evaluation methodologies for terabyte-scale test collections.
SIGIR Forum, 2003

Challenges in information retrieval and language modeling: report of a workshop held at the center for intelligent information retrieval, University of Massachusetts Amherst, September 2002.
SIGIR Forum, 2003

Overview of the TREC 2003 Robust Retrieval Track.
Proceedings of The Twelfth Text REtrieval Conference, 2003

Overview of the TREC 2003 Question Answering Track.
Proceedings of The Twelfth Text REtrieval Conference, 2003

Overview of TREC 2003.
Proceedings of The Twelfth Text REtrieval Conference, 2003

Evaluating Answers to Definition Questions.
Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, 2003

Evaluating the Evaluation: A Case Study Using the TREC 2002 Question Answering Track.
Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, 2003

2002
Overview of the TREC 2002 Question Answering Track.
Proceedings of The Eleventh Text REtrieval Conference, 2002

Overview of TREC 2002.
Proceedings of The Eleventh Text REtrieval Conference, 2002

The effect of topic set size on retrieval experiment error.
Proceedings of the SIGIR 2002: Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2002

2001
Report on TREC-9.
SIGIR Forum, 2001

The TREC question answering track.
Nat. Lang. Eng., 2001

Overview of the TREC 2001 Question Answering Track.
Proceedings of The Tenth Text REtrieval Conference, 2001

Overview of TREC 2001.
Proceedings of The Tenth Text REtrieval Conference, 2001

Evaluation by Highly Relevant Documents.
Proceedings of the SIGIR 2001: Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2001

Philosophy of IR Evaluation.
Proceedings of the Working Notes for CLEF 2001 Workshop co-located with the 5th European Conference on Digital Libraries (ECDL 2001), 2001

The Philosophy of Information Retrieval Evaluation.
Proceedings of the Evaluation of Cross-Language Information Retrieval Systems, 2001

Report from TREC-9.
Proceedings of the Working Notes for CLEF 2001 Workshop co-located with the 5th European Conference on Digital Libraries (ECDL 2001), 2001

Question Answering in TREC.
Proceedings of the 2001 ACM CIKM International Conference on Information and Knowledge Management, 2001

2000
The TREC-5 Confusion Track: Comparing Retrieval Methods for Scanned Text.
Inf. Retr., 2000

Overview of the Sixth Text REtrieval Conference (TREC-6).
Inf. Process. Manag., 2000

Variations in relevance judgments and the measurement of retrieval effectiveness.
Inf. Process. Manag., 2000

The Sixth Text REtrieval Conference (TREC-6).
Inf. Process. Manag., 2000

Overview of the Ninth Text REtrieval Conference (TREC-9).
Proceedings of The Ninth Text REtrieval Conference, 2000

Overview of the TREC-9 Question Answering Track.
Proceedings of The Ninth Text REtrieval Conference, 2000

Spoken Document Retrieval Track Slides.
Proceedings of The Ninth Text REtrieval Conference, 2000

Building a question answering test collection.
Proceedings of the SIGIR 2000: Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2000

The TREC Spoken Document Retrieval Track: A Success Story.
Proceedings of the Computer-Assisted Information Retrieval (Recherche d'Information et ses Applications), 2000

The TREC-8 Question Answering Track.
Proceedings of the Second International Conference on Language Resources and Evaluation, 2000

1999
The Text REtrieval Conference (TREC): History and Plans for TREC-9.
SIGIR Forum, 1999

The TREC-8 Question Answering Track Evaluation.
Proceedings of The Eighth Text REtrieval Conference, 1999

Overview of the Eighth Text REtrieval Conference (TREC-8).
Proceedings of The Eighth Text REtrieval Conference, 1999

The TREC-8 Question Answering Track Report.
Proceedings of The Eighth Text REtrieval Conference, 1999

Overview of the TREC-8 Web Track - Results.
Proceedings of The Eighth Text REtrieval Conference, 1999

Overview of the TREC-8 Web Track.
Proceedings of The Eighth Text REtrieval Conference, 1999

Natural Language Processing and Information Retrieval.
Proceedings of the Information Extraction: Towards Scalable, Adaptable Systems, 1999

1998
Disambiguating Highly Ambiguous Words.
Comput. Linguistics, 1998

The Text Retrieval Conferences (TRECS).
Proceedings of the TIPSTER TEXT PROGRAM PHASE III: Proceedings of a Workshop held at Baltimore, 1998

1997
TREC-6 1997 Spoken Document Retrieval Track Overview and Results.
Proceedings of The Sixth Text REtrieval Conference, 1997

Multiple Search Engines in Database Merging.
Proceedings of the 2nd ACM International Conference on Digital Libraries, 1997

1996
Overview of the Fifth Text REtrieval Conference (TREC-5).
Proceedings of The Fifth Text REtrieval Conference, 1996

The TREC-5 Database Merging Track.
Proceedings of The Fifth Text REtrieval Conference, 1996

Report on the TREC-5 Confusion Track.
Proceedings of The Fifth Text REtrieval Conference, 1996

1995
Siemens TREC-4 Report: Further Experiments with Database Merging.
Proceedings of The Fourth Text REtrieval Conference, 1995

Learning Collection Fusion Strategies.
Proceedings of the SIGIR'95, 1995

Learning Collection Fusion Strategies for Information Retrieval.
Proceedings of the Machine Learning, 1995

1994
The Collection Fusion Problem.
Proceedings of The Third Text REtrieval Conference, 1994

Query Expansion Using Lexical-Semantic Relations.
Proceedings of the 17th Annual International ACM-SIGIR Conference on Research and Development in Information Retrieval. Dublin, 1994

1993
On Expanding Query Vectors with Lexically Related Words.
Proceedings of The Second Text REtrieval Conference, 1993

Using WordNet to Disambiguate Word Senses for Text Retrieval.
Proceedings of the 16th Annual International ACM-SIGIR Conference on Research and Development in Information Retrieval. Pittsburgh, PA, USA, June 27, 1993

Exploiting Concept Spaces for Text Retrieval.
Proceedings of the Human Language Technology: Proceedings of a Workshop Held at Plainsboro, 1993

Corpus-Based Statistical Sense Resolution.
Proceedings of the Human Language Technology: Proceedings of a Workshop Held at Plainsboro, 1993

1992
Vector Expansion in a Large Collection.
Proceedings of The First Text REtrieval Conference, 1992

1989
Intelligent routers.
Proceedings of the 9th International Conference on Distributed Computing Systems, 1989

1986
Implementing agglomerative hierarchic clustering algorithms for use in document retrieval.
Inf. Process. Manag., 1986

The Efficiency of Inverted Index and Cluster Searches.
Proceedings of the SIGIR'86, 1986

1985
The Effectiveness & Efficiency of Agglomerative Hierarchic Clustering in Document Retrieval.
PhD thesis, 1985

Advanced feedback methods in information retrieval.
J. Am. Soc. Inf. Sci., 1985

The Cluster Hypothesis Revisited.
Proceedings of the 8th annual international ACM SIGIR conference on Research and development in information retrieval, 1985

Automatic Assignment of Soft Boolean Operators.
Proceedings of the 8th annual international ACM SIGIR conference on Research and development in information retrieval, 1985

1984
A comparison of two methods for boolean query relevancy feedback.
Inf. Process. Manag., 1984


  Loading...