Abstract
This paper describes a method to quantify the amount of information H(t|s) added by the target sentence t that is not present in the source s in a neural machine translation system. We do this by providing the model the target sentence in a highly compressed form (a “cheat code”), and exploring the effect of the size of the cheat code. We find that the model is able to capture extra information from just a single float representation of the target and nearly reproduces the target with two 32-bit floats per target token.- Anthology ID:
- 2022.naacl-main.177
- Volume:
- Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
- Month:
- July
- Year:
- 2022
- Address:
- Seattle, United States
- Venue:
- NAACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 2472–2477
- Language:
- URL:
- https://aclanthology.org/2022.naacl-main.177
- DOI:
- 10.18653/v1/2022.naacl-main.177
- Cite (ACL):
- Proyag Pal and Kenneth Heafield. 2022. Cheat Codes to Quantify Missing Source Information in Neural Machine Translation. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2472–2477, Seattle, United States. Association for Computational Linguistics.
- Cite (Informal):
- Cheat Codes to Quantify Missing Source Information in Neural Machine Translation (Pal & Heafield, NAACL 2022)
- PDF:
- https://preview.aclanthology.org/auto-file-uploads/2022.naacl-main.177.pdf