Practical Attacks on Machine Translation using Paraphrase

Elizabeth M Merkhofer, John Henderson, Abigail Gertner, Michael Doyle, Lily Wong

Abstract
Studies show machine translation systems are vulnerable to adversarial attacks, where a small change to the input produces an undesirable change in system behavior. This work considers whether this vulnerability exists for attacks crafted with limited information about the target: without access to ground truth references or the particular MT system under attack. It also applies a higher threshold of success, taking into account both source language meaning preservation and target language meaning degradation. We propose an attack that generates edits to an input using a finite state transducer over lexical and phrasal paraphrases and selects one perturbation for meaning preservation and expected degradation of a target system. Attacks against eight state-of-the-art translation systems covering English-German, English-Czech and English-Chinese are evaluated under black-box and transfer scenarios, including cross-language and cross-system transfer. Results suggest that successful single-system attacks seldom transfer across models, especially when crafted without ground truth, but ensembles show promise for generalizing attacks.
Anthology ID:
2022.amta-research.17
Volume:
Proceedings of the 15th biennial conference of the Association for Machine Translation in the Americas (Volume 1: Research Track)
Month:
September
Year:
2022
Address:
Orlando, USA
Editors:
Kevin Duh, Francisco Guzmán
Venue:
AMTA
SIG:
Publisher:
Association for Machine Translation in the Americas
Note:
Pages:
227–239
Language:
URL:
https://aclanthology.org/2022.amta-research.17
DOI:
Bibkey:
Cite (ACL):
Elizabeth M Merkhofer, John Henderson, Abigail Gertner, Michael Doyle, and Lily Wong. 2022. Practical Attacks on Machine Translation using Paraphrase. In Proceedings of the 15th biennial conference of the Association for Machine Translation in the Americas (Volume 1: Research Track), pages 227–239, Orlando, USA. Association for Machine Translation in the Americas.
Cite (Informal):
Practical Attacks on Machine Translation using Paraphrase (Merkhofer et al., AMTA 2022)
Copy Citation:
PDF:
https://preview.aclanthology.org/emnlp22-frontmatter/2022.amta-research.17.pdf
PDF Search