VERTa: Facing a Multilingual Experience of a Linguistically-based MT Evaluation

Elisabet Comelles, Jordi Atserias, Victoria Arranz, Irene Castellón, Jordi Sesé

Abstract
There are several MT metrics used to evaluate translation into Spanish, although most of them use partial or little linguistic information. In this paper we present the multilingual capability of VERTa, an automatic MT metric that combines linguistic information at lexical, morphological, syntactic and semantic level. In the experiments conducted we aim at identifying those linguistic features that prove the most effective to evaluate adequacy in Spanish segments. This linguistic information is tested both as independent modules (to observe what each type of feature provides) and in a combinatory fastion (where different kinds of information interact with each other). This allows us to extract the optimal combination. In addition we compare these linguistic features to those used in previous versions of VERTa aimed at evaluating adequacy for English segments. Finally, experiments show that VERTa can be easily adapted to other languages than English and that its collaborative approach correlates better with human judgements on adequacy than other well-known metrics.
Anthology ID:
L14-1026
Volume:
Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)
Month:
May
Year:
2014
Address:
Reykjavik, Iceland
Editors:
Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Hrafn Loftsson, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association (ELRA)
Note:
Pages:
2701–2707
Language:
URL:
http://www.lrec-conf.org/proceedings/lrec2014/pdf/1032_Paper.pdf
DOI:
Bibkey:
Cite (ACL):
Elisabet Comelles, Jordi Atserias, Victoria Arranz, Irene Castellón, and Jordi Sesé. 2014. VERTa: Facing a Multilingual Experience of a Linguistically-based MT Evaluation. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14), pages 2701–2707, Reykjavik, Iceland. European Language Resources Association (ELRA).
Cite (Informal):
VERTa: Facing a Multilingual Experience of a Linguistically-based MT Evaluation (Comelles et al., LREC 2014)
Copy Citation:
PDF:
http://www.lrec-conf.org/proceedings/lrec2014/pdf/1032_Paper.pdf
PDF Search