ValCAT: Variable-Length Contextualized Adversarial Transformations Using Encoder-Decoder Language Model
Chuyun Deng, Mingxuan Liu, Yue Qin, Jia Zhang, Hai-Xin Duan, Donghong Sun
Abstract
Adversarial texts help explore vulnerabilities in language models, improve model robustness, and explain their working mechanisms. However, existing word-level attack methods trap in a one-to-one attack pattern, i.e., only a single word can be modified in one transformation round, and they ignore the interactions between several consecutive words. In this paper, we propose ValCAT, a black-box attack framework that misleads the language model by applying variable-length contextualized transformations to the original text. Compared to word-level methods, ValCAT expands the basic units of perturbation from single words to spans composed of multiple consecutive words, enhancing the perturbation capability. Experiments show that our method outperforms state-of-the-art methods in terms of attack success rate, perplexity, and semantic similarity on several classification tasks and inference tasks. The comprehensive human evaluation demonstrates that ValCAT has a significant advantage in ensuring the fluency of the adversarial examples and achieves better semantic consistency. We release the code at https://github.com/linerxliner/ValCAT.- Anthology ID:
- 2022.naacl-main.125
- Volume:
- Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
- Month:
- July
- Year:
- 2022
- Address:
- Seattle, United States
- Editors:
- Marine Carpuat, Marie-Catherine de Marneffe, Ivan Vladimir Meza Ruiz
- Venue:
- NAACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1735–1746
- Language:
- URL:
- https://aclanthology.org/2022.naacl-main.125
- DOI:
- 10.18653/v1/2022.naacl-main.125
- Cite (ACL):
- Chuyun Deng, Mingxuan Liu, Yue Qin, Jia Zhang, Hai-Xin Duan, and Donghong Sun. 2022. ValCAT: Variable-Length Contextualized Adversarial Transformations Using Encoder-Decoder Language Model. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1735–1746, Seattle, United States. Association for Computational Linguistics.
- Cite (Informal):
- ValCAT: Variable-Length Contextualized Adversarial Transformations Using Encoder-Decoder Language Model (Deng et al., NAACL 2022)
- PDF:
- https://preview.aclanthology.org/dois-2013-emnlp/2022.naacl-main.125.pdf
- Code
- linerxliner/valcat
- Data
- AG News, GLUE, IMDb Movie Reviews, MultiNLI, QNLI, SNLI