Building a Domain-specific Document Collection for Evaluating Metadata Effects on Information Retrieval

Walid Magdy, Jinming Min, Johannes Leveling, Gareth J. F. Jones


Abstract
This paper describes the development of a structured document collection containing user-generated text and numerical metadata for exploring the exploitation of metadata in information retrieval (IR). The collection consists of more than 61,000 documents extracted from YouTube video pages on basketball in general and NBA (National Basketball Association) in particular, together with a set of 40 topics and their relevance judgements. In addition, a collection of nearly 250,000 user profiles related to the NBA collection is available. Several baseline IR experiments report the effect of using video-associated metadata on retrieval effectiveness. The results surprisingly show that searching the videos titles only performs significantly better than searching additional metadata text fields of the videos such as the tags or the description.
Anthology ID:
L10-1242
Volume:
Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC'10)
Month:
May
Year:
2010
Address:
Valletta, Malta
Venue:
LREC
SIG:
Publisher:
European Language Resources Association (ELRA)
Note:
Pages:
Language:
URL:
http://www.lrec-conf.org/proceedings/lrec2010/pdf/353_Paper.pdf
DOI:
Bibkey:
Cite (ACL):
Walid Magdy, Jinming Min, Johannes Leveling, and Gareth J. F. Jones. 2010. Building a Domain-specific Document Collection for Evaluating Metadata Effects on Information Retrieval. In Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC'10), Valletta, Malta. European Language Resources Association (ELRA).
Cite (Informal):
Building a Domain-specific Document Collection for Evaluating Metadata Effects on Information Retrieval (Magdy et al., LREC 2010)
Copy Citation:
PDF:
http://www.lrec-conf.org/proceedings/lrec2010/pdf/353_Paper.pdf