TinyAttack: Exploring Stylistic Vulnerabilities in Large Language Models

Mamta Mamta, Bogdan Grecu, Oana Cocarascu


Abstract
Large Language Models (LLMs) have demonstrated impressive results in natural language processing (NLP) tasks, however, their brittleness against subtle input perturbations continues to pose a significant challenge. Existing research on robustness has predominantly focused on standard text-based perturbations and the use of invisible characters and homoglyphs, while overlooking the impact of stylized characters increasingly prevalent on social media. To address this, we propose TinyAttack, a novel adversarial attack framework designed to exploit vulnerabilities in LLMs through Unicode-based stylistic transformations. TinyAttack utilises five Unicode variants to modify the visual rendering of text without altering its underlying semantic or syntactic structure. Our comprehensive evaluation on both open-source (Llama, Mistral, Gemma, Qwen) and closed-source LLMs (Gemini, GPT) demonstrates their susceptibility to these stylized inputs, with performance drops ranging from 29-92% and 6-88.5%, respectively, across all tasks.Our code is available at https://github.com/TRAI-group/TinyAttack.
Anthology ID:
2026.findings-acl.1987
Volume:
Findings of the Association for Computational Linguistics: ACL 2026
Month:
July
Year:
2026
Address:
San Diego, California, United States
Editors:
Maria Liakata, Viviane P. Moreira, Jiajun Zhang, David Jurgens
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
39933–39962
Language:
URL:
https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1987/
DOI:
Bibkey:
Cite (ACL):
Mamta Mamta, Bogdan Grecu, and Oana Cocarascu. 2026. TinyAttack: Exploring Stylistic Vulnerabilities in Large Language Models. In Findings of the Association for Computational Linguistics: ACL 2026, pages 39933–39962, San Diego, California, United States. Association for Computational Linguistics.
Cite (Informal):
TinyAttack: Exploring Stylistic Vulnerabilities in Large Language Models (Mamta et al., Findings 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1987.pdf
Checklist:
 2026.findings-acl.1987.checklist.pdf