On the Volatility of Commercial Search Engines and its Impact on Information Retrieval Research

Jimmy, Jimmy and Zuccon, Guido and Demartini, Gianluca (2018) On the Volatility of Commercial Search Engines and its Impact on Information Retrieval Research. In: International ACM SIGIR Conference on Research and Development in Information Retrieval, 8-12 July 2018, Ann Arbor, MI, USA.

PDF Jimmy_2018_On the Volatility_Rev.pdf Download (2MB)

Abstract

We studied the volatility of commercial search engines and reflected on its impact on research that uses them as basis of algorithmical techniques or for user studies. Search engine volatility refers to the fact that a query posed to a search engine at two different points in time returns different documents. By comparing search results retrieved every 2 days over a period of 64 days, we found that the considered commercial search engine API consistently presented volatile search results: it both retrieved new documents, and it ranked documents previously retrieved at different ranks throughout time. Moreover, not only results are volatile: we also found that the effectiveness of the search engine in answering a query is volatile. Our findings reaffirmed that results from commercial search engines are volatile and that care should be taken when using these as basis for researching new information retrieval techniques or performing user studies.

Item Type:	Conference or Workshop Item (Paper)
Subjects:	Z Bibliography. Library Science. Information Resources > Z665 Library Science. Information Science
Divisions:	Faculty of Engineering > Department of Information Technology
Depositing User:	JIMMY 61156 206002
Date Deposited:	10 Feb 2020 02:49
Last Modified:	07 May 2021 06:59
URI:	http://repository.ubaya.ac.id/id/eprint/37281

Actions (login required)

View Item