Bailey, PeterCraswell, NickHawking, David2015-12-132015-12-130306-4573http://hdl.handle.net/1885/85484Past research into text retrieval methods for the Web has been restricted by the lack of a test collection capable of supporting experiments which are both realistic and reproducible. The 1.69 million document WT10g collection is proposed as a multi-purpose testbed for experiments with these attributes, in distributed IR, hyperlink algorithms and conventional ad hoc retrieval. WT10g was constructed by selecting from a superset of documents in such a way that desirable corpus properties were preserved or optimised. These properties include: a high degree of inter-server connectivity, integrity of server holdings, inclusion of documents related to a very wide spread of likely queries, and a realistic distribution of server holding sizes. We confirm that WT10g contains exploitable link information using a site (homepage) finding experiment. Our results show that, on this task, Okapi BM25 works better on propagated link anchor text than on full text. WT10g was used in TREC-9 and TREC-2000 and both topic relevance and homepage finding queries and judgments are available.Keywords: Algorithms; Information retrieval; Optimization; Query languages; Servers; Link-based ranking; World Wide Web Distributed information retrieval; Link-based ranking; Test collections; Web retrieval200310.1016/S0306-4573(02)00084-52015-12-12