Toward Machine Translation Literacy: How Lay Users Perceive and Rely on Imperfect Translations

Yimin Xiao, Yongle Zhang, Dayeon Ki, Calvin Bao, Marianna J. Martindale, Charlotte Vaughn, Ge Gao, Marine Carpuat


Abstract
As Machine Translation (MT) becomes increasingly commonplace, understanding how the general public perceives and relies on imperfect MT is crucial for contextualizing MT research in real-world applications. We present a human study conducted in a public museum (n=452), investigating how fluency and adequacy errors impact bilingual and non-bilingual users’ reliance on MT during casual use. Our findings reveal that non-bilingual users often over-rely on MT due to a lack of evaluation strategies and alternatives, while experiencing the impact of errors can prompt users to reassess future reliance. This highlights the need for MT evaluation and NLP explanation techniques to promote not only MT quality, but also MT literacy among its users.
Anthology ID:
2025.emnlp-main.1725
Volume:
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Month:
November
Year:
2025
Address:
Suzhou, China
Editors:
Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, Violet Peng
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
33985–34002
Language:
URL:
https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-main.1725/
DOI:
Bibkey:
Cite (ACL):
Yimin Xiao, Yongle Zhang, Dayeon Ki, Calvin Bao, Marianna J. Martindale, Charlotte Vaughn, Ge Gao, and Marine Carpuat. 2025. Toward Machine Translation Literacy: How Lay Users Perceive and Rely on Imperfect Translations. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 33985–34002, Suzhou, China. Association for Computational Linguistics.
Cite (Informal):
Toward Machine Translation Literacy: How Lay Users Perceive and Rely on Imperfect Translations (Xiao et al., EMNLP 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-main.1725.pdf
Checklist:
 2025.emnlp-main.1725.checklist.pdf