Abstract
We study semantic collisions: texts that are semantically unrelated but judged as similar by NLP models. We develop gradient-based approaches for generating semantic collisions and demonstrate that state-of-the-art models for many tasks which rely on analyzing the meaning and similarity of texts—including paraphrase identification, document retrieval, response suggestion, and extractive summarization—are vulnerable to semantic collisions. For example, given a target query, inserting a crafted collision into an irrelevant document can shift its retrieval rank from 1000 to top 3. We show how to generate semantic collisions that evade perplexity-based filtering and discuss other potential mitigations. Our code is available at https://github.com/csong27/collision-bert.- Anthology ID:
- 2020.emnlp-main.344
- Volume:
- Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
- Month:
- November
- Year:
- 2020
- Address:
- Online
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 4198–4210
- Language:
- URL:
- https://aclanthology.org/2020.emnlp-main.344
- DOI:
- 10.18653/v1/2020.emnlp-main.344
- Cite (ACL):
- Congzheng Song, Alexander Rush, and Vitaly Shmatikov. 2020. Adversarial Semantic Collisions. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4198–4210, Online. Association for Computational Linguistics.
- Cite (Informal):
- Adversarial Semantic Collisions (Song et al., EMNLP 2020)
- PDF:
- https://preview.aclanthology.org/starsem-semeval-split/2020.emnlp-main.344.pdf
- Code
- csong27/collision-bert
- Data
- MRPC