@inproceedings{li-etal-2025-llms-caught,
    title = "{LLM}s Caught in the Crossfire: Malware Requests and Jailbreak Challenges",
    author = "Li, Haoyang  and
      Gao, Huan  and
      Zhao, Zhiyuan  and
      Lin, Zhiyu  and
      Gao, Junyu  and
      Li, Xuelong",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingestion-acl-25/2025.acl-long.1350/",
    pages = "27833--27848",
    ISBN = "979-8-89176-251-0",
    abstract = "The widespread adoption of Large Language Models (LLMs) has heightened concerns about their security, particularly their vulnerability to jailbreak attacks that leverage crafted prompts to generate malicious outputs. While prior research has been conducted on general security capabilities of LLMs, their specific susceptibility to jailbreak attacks in code generation remains largely unexplored. To fill this gap, we propose MalwareBench, a benchmark dataset containing 3,520 jailbreaking prompts for malicious code-generation, designed to evaluate LLM robustness against such threats. MalwareBench is based on 320 manually crafted malicious code generation requirements, covering 11 jailbreak methods and 29 code functionality categories. Experiments show that mainstream LLMs exhibit limited ability to reject malicious code-generation requirements, and the combination of multiple jailbreak methods further reduces the model{'}s security capabilities: specifically, the average rejection rate for malicious content is 60.93{\%}, dropping to 39.92{\%} when combined with jailbreak attack algorithms. Our work highlights that the code security capabilities of LLMs still pose significant challenges."
}