Abstract
Much of the work on Statistical Machine Translation (SMT) from morphologically rich languages has shown that morphological tokenization and orthographic normalization help improve SMT quality because of the sparsity reduction they contribute. In this paper, we study the effect of these processes on SMT when translating into a morphologically rich language, namely Arabic.We explore a space of tokenization schemes and normalization options. We only evaluate on detokenized and orthographically correct (enriched) output. Our results show that the best performing tokenization scheme is that of the Penn Arabic Treebank. Additionally, training on orthographically normalized (reduced) text then jointly enriching and detokenizing the output outperforms training on enriched text.- Anthology ID:
- 2010.jeptalnrecital-long.29
- Volume:
- Actes de la 17e conférence sur le Traitement Automatique des Langues Naturelles. Articles longs
- Month:
- July
- Year:
- 2010
- Address:
- Montréal, Canada
- Venue:
- JEP/TALN/RECITAL
- SIG:
- Publisher:
- ATALA
- Note:
- Pages:
- 282–291
- Language:
- URL:
- https://aclanthology.org/2010.jeptalnrecital-long.29
- DOI:
- Cite (ACL):
- Ahmed El Kholy and Nizar Habash. 2010. Orthographic and Morphological Processing for English-Arabic Statistical Machine Translation. In Actes de la 17e conférence sur le Traitement Automatique des Langues Naturelles. Articles longs, pages 282–291, Montréal, Canada. ATALA.
- Cite (Informal):
- Orthographic and Morphological Processing for English-Arabic Statistical Machine Translation (El Kholy & Habash, JEP/TALN/RECITAL 2010)
- PDF:
- https://preview.aclanthology.org/auto-file-uploads/2010.jeptalnrecital-long.29.pdf