@inproceedings{peng-etal-2025-pwngpt,
    title = "{P}wn{GPT}: Automatic Exploit Generation Based on Large Language Models",
    author = "Peng, Wanzong  and
      Ye, Lin  and
      Du, Xuetao  and
      Zhang, Hongli  and
      Zhan, Dongyang  and
      Zhang, Yunting  and
      Guo, Yicheng  and
      Zhang, Chen",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingestion-acl-25/2025.acl-long.562/",
    pages = "11481--11494",
    ISBN = "979-8-89176-251-0",
    abstract = "Automatic exploit generation (AEG) refers to the automatic discovery and exploitation of vulnerabilities against unknown targets. Traditional AEG often targets a single type of vulnerability and still relies on templates built from expert experience. To achieve intelligent exploit generation, we establish a comprehensive benchmark using Binary Exploitation (pwn) challenges in Capture the Flag (CTF) competitions and investigate the capabilities of Large Language Models (LLMs) in AEG based on the benchmark. To improve the performance of AEG, we propose PwnGPT, an LLM-based automatic exploit generation framework that automatically solves pwn challenges. The structural design of PwnGPT is divided into three main components: analysis, generation, and verification modules. With the help of a modular approach and structured problem inputs, PwnGPT can solve challenges that LLMs cannot directly solve. We evaluate PwnGPT on our benchmark and analyze the outputs of each module. Experimental results show that our framework is highly autonomous and capable of addressing various challenges. Compared to direct input LLMs, PwnGPT increases the completion rate of exploit on our benchmark from 26.3{\%} to 57.9{\%} with the OpenAI o1-preview model and from 21.1{\%} to 36.8{\%} with the GPT-4o model."
}