Skip navigation
Skip navigation

Workload sampling for enterprise search evaluation

Rowlands, Thomas; Hawking, David; Sankaranarayana, Ramesh S


In real world use of test collection methods, it is essential that the query test set be representative of the work load expected in the actual application. Using a random sample of queries from a media company's query log as a 'gold standard' test set we demonstrate that biases in sitemap-derived and top n query sets can lead to significant perturbations in engine rankings and big differences in estimated performance levels.

CollectionsANU Research Publications
Date published: 2007
Type: Conference paper
Source: Proceedings of 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval
DOI: 10.1145/1277741.1277959


File Description SizeFormat Image
01_Rowlands_Workload_sampling_for_2007.pdf217.9 kBAdobe PDF    Request a copy

Items in Open Research are protected by copyright, with all rights reserved, unless otherwise indicated.

Updated:  19 May 2020/ Responsible Officer:  University Librarian/ Page Contact:  Library Systems & Web Coordinator