Q-FAKER: Query-free Hard Black-box Attack via Controlled Generation

CheolWon Na, YunSeok Choi, Jee-Hyong Lee


Abstract
Many adversarial attack approaches are proposed to verify the vulnerability of language models. However, they require numerous queries and the information on the target model. Even black-box attack methods also require the target model’s output information. They are not applicable in real-world scenarios, as in hard black-box settings where the target model is closed and inaccessible. Even the recently proposed hard black-box attacks still require many queries and demand extremely high costs for training adversarial generators. To address these challenges, we propose Q-faker (Query-free Hard Black-box Attacker), a novel and efficient method that generates adversarial examples without accessing the target model. To avoid accessing the target model, we use a surrogate model instead. The surrogate model generates adversarial sentences for a target-agnostic attack. During this process, we leverage controlled generation techniques. We evaluate our proposed method on eight datasets. Experimental results demonstrate our method’s effectiveness including high transferability and the high quality of the generated adversarial examples, and prove its practical in hard black-box settings.
Anthology ID:
2025.findings-naacl.463
Volume:
Findings of the Association for Computational Linguistics: NAACL 2025
Month:
April
Year:
2025
Address:
Albuquerque, New Mexico
Editors:
Luis Chiruzzo, Alan Ritter, Lu Wang
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
8274–8289
Language:
URL:
https://preview.aclanthology.org/fix-sig-urls/2025.findings-naacl.463/
DOI:
Bibkey:
Cite (ACL):
CheolWon Na, YunSeok Choi, and Jee-Hyong Lee. 2025. Q-FAKER: Query-free Hard Black-box Attack via Controlled Generation. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 8274–8289, Albuquerque, New Mexico. Association for Computational Linguistics.
Cite (Informal):
Q-FAKER: Query-free Hard Black-box Attack via Controlled Generation (Na et al., Findings 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/fix-sig-urls/2025.findings-naacl.463.pdf