Overview of the CLEF 2018 Consumer Health Search Task

Jimmy, Jimmy and Zuccon, Guido and Palotti, Joao and Goeuriot, Lorraine and Kelly, Liadh (2018) Overview of the CLEF 2018 Consumer Health Search Task. In: Conference and Labs of the Evaluation Forum (CLEF), 10-14 September 2018, Avignon, France.

PDF JImmy_2018_Overview CLEF2018.pdf Download (1MB)

Abstract

This paper details the collection, systems and evaluation methods used in the CLEF 2018 eHealth Evaluation Lab, Consumer Health Search (CHS) task (Task 3). This task investigates the effectiveness of search engines in providing access to medical information present on the Web for people that have no or little medical knowledge. The task aims to foster advances in the development of search technologies for Consumer Health Search by providing resources and evaluation methods to test and validate search systems. Built upon the the 2013-17 series of CLEF eHealth Information Retrieval tasks, the 2018 task considers both mono- and multilingual retrieval, embracing the Text REtrieval Conference (TREC) -style evaluation process with a shared collection of documents and queries, the contribution of runs from participants and the subsequent formation of relevance assessments and evaluation of the participants submissions. For this year, the CHS task uses a new Web corpus and a new set of queries compared to the previous years. The new corpus consists of Web pages acquired from the CommonCrawl and the new set of queries consists of 50 queries issued by the general public to the Health on the Net (HON) search services. We then manually translated the 50 queries to French, German, and Czech; and obtained English query variations of the 50 original queries. A total of 7 teams from 7 different countries participated in the 2018 CHS task: CUNI (Czech Republic), IMS Unipd (Italy), MIRACL (Tunisia), QUT (Australia), SINAI (Spain), UB-Botswana (Botswana), and UEvora (Portugal).

Item Type:	Conference or Workshop Item (Paper)
Uncontrolled Keywords:	Evaluation, Consumer Health Search, Medical Information Retrieval
Subjects:	Z Bibliography. Library Science. Information Resources > Z665 Library Science. Information Science
Divisions:	Faculty of Engineering > Department of Information Technology
Depositing User:	JIMMY 61156 206002
Date Deposited:	10 Feb 2020 02:47
Last Modified:	06 May 2021 03:14
URI:	http://repository.ubaya.ac.id/id/eprint/37282

Actions (login required)

View Item