@inproceedings{mei-etal-2026-good,
    title = "Good Reasoning Makes Good Demonstrations: Implicit Reasoning Quality Supervision via In-Context Reinforcement Learning",
    author = "Mei, Tiehua  and
      Lv, Minxuan  and
      Pan, Leiyu  and
      Su, Zhenpeng  and
      Hou, Hongru  and
      Chen, Hengrui  and
      Xu, Ao  and
      Yang, Deqing",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1523/",
    pages = "30452--30469",
    ISBN = "979-8-89176-395-1",
    abstract = "Reinforcement Learning with Verifiable Rewards (RLVR) improves reasoning in large language models but treats all correct solutions equally, potentially reinforcing flawed traces that arrive at correct answers by chance. We observe that \textit{better reasoning makes better demonstrations}: high-quality solutions serve as more effective in-context examples than low-quality ones. We term this teaching ability \textbf{Demonstration Utility}, and show that the policy model{'}s own in-context learning ability provides an efficient way to measure it, yielding a quality signal termed \textbf{Evidence Gain}. To leverage this signal during training, we introduce \textbf{In-Context RLVR}, which prepends demonstrations before each rollout. Theoretically, we prove that this simple input modification implicitly reweights rewards by a factor approximately proportional to Evidence Gain, assigning higher weights to high-quality traces without requiring costly computation. Experiments on mathematical reasoning benchmarks demonstrate consistent improvements in both accuracy and reasoning quality over standard RLVR baselines. Our codes and datasets are available at \url{https://github.com/Mithas-114/IC-DAPO}."
}