Págico: Evaluating Wikipedia-based information retrieval in Portuguese

Cristina Mota, Alberto Simões, Cláudia Freitas, Luís Costa, Diana Santos

Abstract
How do people behave in their everyday information seeking tasks, which often involve Wikipedia? Are there systems which can help them, or do a similar job? In this paper we describe Págico, an evaluation contest with the main purpose of fostering research in these topics. We describe its motivation, the collection of documents created, the evaluation setup, the topics chosen and their choice, the participation, as well as the measures used for evaluation and the gathered resources. The task―between information retrieval and question answering―can be further described as answering questions related to Portuguese-speaking culture in the Portuguese Wikipedia, in a number of different themes and geographic and temporal angles. This initiative allowed us to create interesting datasets and perform some assessment of Wikipedia, while also improving a public-domain open-source system for further wikipedia-based evaluations. In the paper, we provide examples of questions, we report the results obtained by the participants, and provide some discussion on complex issues.
Anthology ID:
L12-1336
Volume:
Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12)
Month:
May
Year:
2012
Address:
Istanbul, Turkey
Editors:
Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Mehmet Uğur Doğan, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association (ELRA)
Note:
Pages:
2015–2022
Language:
URL:
http://www.lrec-conf.org/proceedings/lrec2012/pdf/590_Paper.pdf
DOI:
Bibkey:
Cite (ACL):
Cristina Mota, Alberto Simões, Cláudia Freitas, Luís Costa, and Diana Santos. 2012. Págico: Evaluating Wikipedia-based information retrieval in Portuguese. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12), pages 2015–2022, Istanbul, Turkey. European Language Resources Association (ELRA).
Cite (Informal):
Págico: Evaluating Wikipedia-based information retrieval in Portuguese (Mota et al., LREC 2012)
Copy Citation:
PDF:
http://www.lrec-conf.org/proceedings/lrec2012/pdf/590_Paper.pdf
PDF Search