@inproceedings{pan-etal-2026-coverrl,
    title = "{C}o{V}er{RL}: Breaking the Consensus Trap in Label-Free Reasoning via Generator-Verifier Co-Evolution",
    author = "Pan, Teng  and
      Yan, Yuchen  and
      Wang, Zixuan  and
      Zhang, Ruiqing  and
      Hou, Guiyang  and
      Zhang, Wenqi  and
      Lu, Weiming  and
      Xiao, Jun  and
      Shen, Yongliang",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.1376/",
    pages = "29833--29853",
    ISBN = "979-8-89176-390-6",
    abstract = "Label-free reinforcement learning enables large language models to improve reasoning capabilities without ground-truth supervision, typically by treating majority-voted answers as pseudo-labels. However, we identify a critical failure mode: as training maximizes self-consistency, output diversity collapses, causing the model to confidently reinforce systematic errors that evade detection. We term this the consensus trap. To escape it, we propose CoVerRL, a framework where a single model alternates between generator and verifier roles, with each capability bootstrapping the other. Majority voting provides noisy but informative supervision for training the verifier, while the improving verifier progressively filters self-consistent errors from pseudo-labels. This co-evolution creates a virtuous cycle that maintains high reward accuracy throughout training. Experiments across Qwen and Llama model families demonstrate that CoVerRL outperforms label-free baselines by 4.7-5.9{\%} on mathematical reasoning benchmarks. Moreover, self-verification accuracy improves from around 55{\%} to over 85{\%}, confirming that both capabilities genuinely co-evolve."
}