Abstract
In this paper, we present a study on the impact of so-called multiword expressions (MWEs) and multiword named entities (NEs) on the performance of Chinese-English machine translation (MT) systems. Built on an extended version of the data from the WMT22 Metrics Shared Task (with extra labels of 9 types of Chinese MWEs, and 19 types of Chinese multiword NEs) which includes scores and error annotations provided by human experts, we make further extraction of MWE- and NE-related translation errors. By investigating the human evaluation scores and the error rates on each category of MWEs and NEs, we find that: 1) MT systems tend to perform significantly worse on Chinese sentences with most kinds of MWEs and NEs; 2) MWEs and NEs which make up of about twenty percent of tokens, i.e. characters in Chinese, result in one-third of translation errors; 3) for 13 categories of MWEs and NEs, the error rates exceed 50% with the highest to be 84.8%. Based on the results, we emphasize that MWEs and NEs are still a bottleneck issue for MT and special attention to MWEs and NEs should be paid to further improving the performance of MT systems.- Anthology ID:
- 2024.findings-emnlp.357
- Volume:
- Findings of the Association for Computational Linguistics: EMNLP 2024
- Month:
- November
- Year:
- 2024
- Address:
- Miami, Florida, USA
- Editors:
- Yaser Al-Onaizan, Mohit Bansal, Yun-Nung Chen
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 6154–6165
- Language:
- URL:
- https://preview.aclanthology.org/add_missing_videos/2024.findings-emnlp.357/
- DOI:
- 10.18653/v1/2024.findings-emnlp.357
- Cite (ACL):
- Huacheng Song and Hongzhi Xu. 2024. A Deep Analysis of the Impact of Multiword Expressions and Named Entities on Chinese-English Machine Translations. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 6154–6165, Miami, Florida, USA. Association for Computational Linguistics.
- Cite (Informal):
- A Deep Analysis of the Impact of Multiword Expressions and Named Entities on Chinese-English Machine Translations (Song & Xu, Findings 2024)
- PDF:
- https://preview.aclanthology.org/add_missing_videos/2024.findings-emnlp.357.pdf