Abstract
This paper reports on a pilot experiment that compares two different machine translation (MT) paradigms in reading comprehension tests. To explore a suitable methodology, we set up a pilot experiment with a group of six users (with English, Spanish and Simplified Chinese languages) using an English Language Testing System (IELTS), and an eye-tracker. The users were asked to read three texts in their native language: either the original English text (for the English speakers) or the machine-translated text (for the Spanish and Simplified Chinese speakers). The original texts were machine-translated via two MT systems: neural (NMT) and statistical (SMT). The users were also asked to rank satisfaction statements on a 3-point scale after reading each text and answering the respective comprehension questions. After all tasks were completed, a post-task retrospective interview took place to gather qualitative data. The findings suggest that the users from the target languages completed more tasks in less time with a higher level of satisfaction when using translations from the NMT system.- Anthology ID:
- 2018.eamt-main.8
- Volume:
- Proceedings of the 21st Annual Conference of the European Association for Machine Translation
- Month:
- May
- Year:
- 2018
- Address:
- Alicante, Spain
- Editors:
- Juan Antonio Pérez-Ortiz, Felipe Sánchez-Martínez, Miquel Esplà-Gomis, Maja Popović, Celia Rico, André Martins, Joachim Van den Bogaert, Mikel L. Forcada
- Venue:
- EAMT
- SIG:
- Publisher:
- Note:
- Pages:
- 99–108
- Language:
- URL:
- https://aclanthology.org/2018.eamt-main.8
- DOI:
- Cite (ACL):
- Sheila Castilho and Ana Guerberof Arenas. 2018. Reading Comprehension of Machine Translation Output: What Makes for a Better Read?. In Proceedings of the 21st Annual Conference of the European Association for Machine Translation, pages 99–108, Alicante, Spain.
- Cite (Informal):
- Reading Comprehension of Machine Translation Output: What Makes for a Better Read? (Castilho & Guerberof Arenas, EAMT 2018)
- PDF:
- https://preview.aclanthology.org/proper-vol2-ingestion/2018.eamt-main.8.pdf