@inproceedings{saiem-etal-2025-sequentialbreak,
    title = "{S}equential{B}reak: Large Language Models Can be Fooled by Embedding Jailbreak Prompts into Sequential Prompt Chains",
    author = "Saiem, Bijoy Ahmed  and
      Shanto, MD Sadik Hossain  and
      Ahsan, Rakib  and
      Rashid, Md Rafi Ur",
    editor = "Zhao, Jin  and
      Wang, Mingyang  and
      Liu, Zhu",
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/landing_page/2025.acl-srw.37/",
    pages = "548--579",
    ISBN = "979-8-89176-254-1",
    abstract = "As the use of Large Language Models (LLMs) expands, so do concerns about their vulnerability to jailbreak attacks. We introduce SequentialBreak, a novel single-query jailbreak technique that arranges multiple benign prompts in sequence with a hidden malicious instruction among them to bypass safety mechanisms. Sequential prompt chains in a single query can lead LLMs to focus on certain prompts while ignoring others. By embedding a malicious prompt within a prompt chain, we show that LLMs tend to ignore the harmful context and respond to all prompts including the harmful one. We demonstrate the effectiveness of our attack across diverse scenarios{---}including Q{\&}A systems, dialogue completion tasks, and levelwise gaming scenario{---}highlighting its adaptability to varied prompt structures. The variability of prompt structures shows that SequentialBreak is adaptable to formats beyond those discussed here. Experiments show that SequentialBreak only uses a single query to significantly outperform existing baselines on both open-source and closed-source models. These findings underline the urgent need for more robust defenses against prompt-based attacks. The Results and website are available on https://anonymous.4open.science/r/JailBreakAttack-4F3B/."
}