@inproceedings{mou-etal-2026-thinking,
    title = "Thinking Twice Makes Large Language Models Safer and More Helpful",
    author = "Mou, Yutao  and
      Luo, Yuxiao  and
      Zhang, Shikun  and
      Ye, Wei",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1812/",
    pages = "36365--36389",
    ISBN = "979-8-89176-395-1",
    abstract = "Current safety alignment techniques for large language models (LLMs) struggle to balance harmlessness and helpfulness: improving safety often comes at the cost of degraded utility. Our preliminary study shows that guiding unaligned base models with safety-aware reasoning that includes explicit self-reflection can effectively defend jailbreak attacks while preserving response quality. This observation motivates internalizing and strengthening self-reflective reasoning capabilities within LLMs to achieve a better safety{--}utility trade-off. We propose Safety-aware Reflective Reasoning Optimization (SaRO), a two-stage framework: (1) Reasoning-style Warmup (RW) to internalize self-reflective reasoning, and (2) Self-reflective Reasoning Process Optimization (SRPO) to encourage reflection and correction. Experiments show that SaRO outperforms existing reasoning-based alignment methods, achieving a better balance of safety and helpfulness."
}