Abstract
Incremental domain adaptation, in which a system learns from the correct output for each input immediately after making its prediction for that input, can dramatically improve system performance for interactive machine translation. Users of interactive systems are sensitive to the speed of adaptation and how often a system repeats mistakes, despite being corrected. Adaptation is most commonly assessed using corpus-level BLEU- or TER-derived metrics that do not explicitly take adaptation speed into account. We find that these metrics often do not capture immediate adaptation effects, such as zero-shot and one-shot learning of domain-specific lexical items. To this end, we propose new metrics that directly evaluate immediate adaptation performance for machine translation. We use these metrics to choose the most suitable adaptation method from a range of different adaptation techniques for neural machine translation systems.- Anthology ID:
- N19-1206
- Volume:
- Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)
- Month:
- June
- Year:
- 2019
- Address:
- Minneapolis, Minnesota
- Editors:
- Jill Burstein, Christy Doran, Thamar Solorio
- Venue:
- NAACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 2038–2046
- Language:
- URL:
- https://aclanthology.org/N19-1206
- DOI:
- 10.18653/v1/N19-1206
- Cite (ACL):
- Patrick Simianer, Joern Wuebker, and John DeNero. 2019. Measuring Immediate Adaptation Performance for Neural Machine Translation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2038–2046, Minneapolis, Minnesota. Association for Computational Linguistics.
- Cite (Informal):
- Measuring Immediate Adaptation Performance for Neural Machine Translation (Simianer et al., NAACL 2019)
- PDF:
- https://preview.aclanthology.org/autopr/N19-1206.pdf