Findings of the WMT 2020 Shared Task on Chat Translation
M. Amin Farajian, António V. Lopes, André F. T. Martins, Sameen Maruf, Gholamreza Haffari
Abstract
We report the results of the first edition of the WMT shared task on chat translation. The task consisted of translating bilingual conversational text, in particular customer support chats for the English-German language pair (English agent, German customer). This task varies from the other translation shared tasks, i.e. news and biomedical, mainly due to the fact that the conversations are bilingual, less planned, more informal, and often ungrammatical. Furthermore, such conversations are usually characterized by shorter and simpler sentences and contain more pronouns. We received 14 submissions from 6 participating teams, all of them covering both directions, i.e. En->De for agent utterances and De->En for customer messages. We used automatic metrics (BLEU and TER) for evaluating the translations of both agent and customer messages and human document-level direct assessments (DDA) to evaluate the agent translations.- Anthology ID:
- 2020.wmt-1.3
- Volume:
- Proceedings of the Fifth Conference on Machine Translation
- Month:
- November
- Year:
- 2020
- Address:
- Online
- Venue:
- WMT
- SIG:
- SIGMT
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 65–75
- Language:
- URL:
- https://aclanthology.org/2020.wmt-1.3
- DOI:
- Cite (ACL):
- M. Amin Farajian, António V. Lopes, André F. T. Martins, Sameen Maruf, and Gholamreza Haffari. 2020. Findings of the WMT 2020 Shared Task on Chat Translation. In Proceedings of the Fifth Conference on Machine Translation, pages 65–75, Online. Association for Computational Linguistics.
- Cite (Informal):
- Findings of the WMT 2020 Shared Task on Chat Translation (Farajian et al., WMT 2020)
- PDF:
- https://preview.aclanthology.org/ingestion-script-update/2020.wmt-1.3.pdf
- Code
- Unbabel/BConTrasT
- Data
- Taskmaster-1