Effects of different types of noise in user-generated reviews on human and machine translations including ChatGPT

Maja Popovic, Ekaterina Lapshinova-Koltunski, Maarit Koponen

Abstract
This paper investigates effects of noisy source texts (containing spelling and grammar errors, informal words or expressions, etc.) on human and machine translations, namely whether the noisy phenomena are kept in the translations, corrected, or caused errors. The analysed data consists of English user reviews of Amazon products translated into Croatian, Russian and Finnish by professional translators, translation students, machine translation (MT) systems, and ChatGPT language model. The results show that overall, ChatGPT and professional translators mostly correct/standardise those parts, while students are often keeping them. Furthermore, MT systems are most prone to errors while ChatGPT is more robust, but notably less robust than human translators. Finally, some of the phenomena are particularly challenging both for MT systems and for ChatGPT, especially spelling errors and informal constructions.
Anthology ID:
2024.wnut-1.3
Volume:
Proceedings of the Ninth Workshop on Noisy and User-generated Text (W-NUT 2024)
Month:
March
Year:
2024
Address:
San Ġiljan, Malta
Editors:
Rob van der Goot, JinYeong Bak, Max Müller-Eberstein, Wei Xu, Alan Ritter, Tim Baldwin
Venues:
WNUT | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
17–30
Language:
URL:
https://aclanthology.org/2024.wnut-1.3
DOI:
Bibkey:
Cite (ACL):
Maja Popovic, Ekaterina Lapshinova-Koltunski, and Maarit Koponen. 2024. Effects of different types of noise in user-generated reviews on human and machine translations including ChatGPT. In Proceedings of the Ninth Workshop on Noisy and User-generated Text (W-NUT 2024), pages 17–30, San Ġiljan, Malta. Association for Computational Linguistics.
Cite (Informal):
Effects of different types of noise in user-generated reviews on human and machine translations including ChatGPT (Popovic et al., WNUT-WS 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/emnlp22-frontmatter/2024.wnut-1.3.pdf
PDF Search