@inproceedings{li-etal-2026-mitigating,
    title = "Mitigating Lost in Multi-turn Conversation via Curriculum {RL} with Verifiable Accuracy and Abstention Rewards",
    author = "Li, Ming  and
      Chen, Pei  and
      Zhang, Zhenhao  and
      Yang, Tao  and
      Zhang, Xinyang  and
      Li, Han  and
      Cao, Tianyu  and
      Zeng, Ming  and
      Wu, Zhuofeng  and
      Jiang, Meng  and
      Li, Huasheng  and
      Li, Lihong  and
      Yin, Bing",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.1540/",
    pages = "33364--33381",
    ISBN = "979-8-89176-390-6",
    abstract = "Large Language Models demonstrate strong capabilities in single-turn instruction following but suffer from Lost-in-Conversation (LiC), a degradation in performance as information is revealed progressively in multi-turn settings. Motivated by the current progress on Reinforcement Learning with Verifiable Rewards (RLVR), we propose Curriculum Reinforcement Learning with Verifiable Accuracy and Abstention Rewards (RLAAR), a framework that encourages models not only to generate correct answers, but also to judge the solvability of questions in the multi-turn conversation setting. Our approach employs a competence-gated curriculum that incrementally increases dialogue difficulty (in terms of instruction shards), stabilizing training while promoting reliability. Using multi-turn, on-policy rollouts and a mixed-reward system, RLAAR teaches models to balance problem-solving with informed abstention, reducing premature answering behaviors that cause LiC. Evaluated on LiC benchmarks, RLAAR significantly mitigates LiC performance decay (62.6{\%} to 75.1{\%}) and improves calibrated abstention rates (33.5{\%} to 73.4{\%}). Together, these results provide a practical recipe for building multi-turn reliable and trustworthy LLMs."
}