Findings of the 2017 DiscoMT Shared Task on Cross-lingual Pronoun Prediction

Sharid Loáiciga, Sara Stymne, Preslav Nakov, Christian Hardmeier, Jörg Tiedemann, Mauro Cettolo, Yannick Versley


Abstract
We describe the design, the setup, and the evaluation results of the DiscoMT 2017 shared task on cross-lingual pronoun prediction. The task asked participants to predict a target-language pronoun given a source-language pronoun in the context of a sentence. We further provided a lemmatized target-language human-authored translation of the source sentence, and automatic word alignments between the source sentence words and the target-language lemmata. The aim of the task was to predict, for each target-language pronoun placeholder, the word that should replace it from a small, closed set of classes, using any type of information that can be extracted from the entire document. We offered four subtasks, each for a different language pair and translation direction: English-to-French, English-to-German, German-to-English, and Spanish-to-English. Five teams participated in the shared task, making submissions for all language pairs. The evaluation results show that most participating teams outperformed two strong n-gram-based language model-based baseline systems by a sizable margin.
Anthology ID:
W17-4801
Volume:
Proceedings of the Third Workshop on Discourse in Machine Translation
Month:
September
Year:
2017
Address:
Copenhagen, Denmark
Editors:
Bonnie Webber, Andrei Popescu-Belis, Jörg Tiedemann
Venue:
DiscoMT
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1–16
Language:
URL:
https://aclanthology.org/W17-4801
DOI:
10.18653/v1/W17-4801
Bibkey:
Cite (ACL):
Sharid Loáiciga, Sara Stymne, Preslav Nakov, Christian Hardmeier, Jörg Tiedemann, Mauro Cettolo, and Yannick Versley. 2017. Findings of the 2017 DiscoMT Shared Task on Cross-lingual Pronoun Prediction. In Proceedings of the Third Workshop on Discourse in Machine Translation, pages 1–16, Copenhagen, Denmark. Association for Computational Linguistics.
Cite (Informal):
Findings of the 2017 DiscoMT Shared Task on Cross-lingual Pronoun Prediction (Loáiciga et al., DiscoMT 2017)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-3/W17-4801.pdf
Attachment:
 W17-4801.Attachment.zip
Data
Europarl