Machine Translation Quality: A comparative evaluation of SMT, NMT and tailored-NMT outputs
Maria Stasimioti, Vilelmini Sosoni, Katia Kermanidis, Despoina Mouratidis
Abstract
The present study aims to compare three systems: a generic statistical machine translation (SMT), a generic neural machine translation (NMT) and a tailored-NMT system focusing on the English to Greek language pair. The comparison is carried out following a mixed-methods approach, i.e. automatic metrics, as well as side-by-side ranking, adequacy and fluency rating, measurement of actual post editing (PE) effort and human error analysis performed by 16 postgraduate Translation students. The findings reveal a higher score for both the generic NMT and the tailored-NMT outputs as regards automatic metrics and human evaluation metrics, with the tailored-NMT output faring even better than the generic NMT output.- Anthology ID:
- 2020.eamt-1.47
- Volume:
- Proceedings of the 22nd Annual Conference of the European Association for Machine Translation
- Month:
- November
- Year:
- 2020
- Address:
- Lisboa, Portugal
- Venue:
- EAMT
- SIG:
- Publisher:
- European Association for Machine Translation
- Note:
- Pages:
- 441–450
- Language:
- URL:
- https://aclanthology.org/2020.eamt-1.47
- DOI:
- Cite (ACL):
- Maria Stasimioti, Vilelmini Sosoni, Katia Kermanidis, and Despoina Mouratidis. 2020. Machine Translation Quality: A comparative evaluation of SMT, NMT and tailored-NMT outputs. In Proceedings of the 22nd Annual Conference of the European Association for Machine Translation, pages 441–450, Lisboa, Portugal. European Association for Machine Translation.
- Cite (Informal):
- Machine Translation Quality: A comparative evaluation of SMT, NMT and tailored-NMT outputs (Stasimioti et al., EAMT 2020)
- PDF:
- https://preview.aclanthology.org/starsem-semeval-split/2020.eamt-1.47.pdf