Development and Application of a Cross-language Document Comparability Metric

Fangzhong Su, Bogdan Babych


Abstract
In this paper we present a metric that measures comparability of documents across different languages. The metric is developed within the FP7 ICT ACCURAT project, as a tool for aligning comparable corpora on the document level; further these aligned comparable documents are used for phrase alignment and extraction of translation equivalents, with the aim to extend phrase tables of statistical MT systems without the need to use parallel texts. The metric uses several features, such as lexical information, document structure, keywords and named entities, which are combined in an ensemble manner. We present the results by measuring the reliability and effectiveness of the metric, and demonstrate its application and the impact for the task of parallel phrase extraction from comparable corpora.
Anthology ID:
L12-1473
Volume:
Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12)
Month:
May
Year:
2012
Address:
Istanbul, Turkey
Venue:
LREC
SIG:
Publisher:
European Language Resources Association (ELRA)
Note:
Pages:
3956–3962
Language:
URL:
http://www.lrec-conf.org/proceedings/lrec2012/pdf/804_Paper.pdf
DOI:
Bibkey:
Cite (ACL):
Fangzhong Su and Bogdan Babych. 2012. Development and Application of a Cross-language Document Comparability Metric. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12), pages 3956–3962, Istanbul, Turkey. European Language Resources Association (ELRA).
Cite (Informal):
Development and Application of a Cross-language Document Comparability Metric (Su & Babych, LREC 2012)
Copy Citation:
PDF:
http://www.lrec-conf.org/proceedings/lrec2012/pdf/804_Paper.pdf