@inproceedings{he-etal-2026-vane,
    title = "{VANE}: Guiding High-Value Exploration in {RLVR} via Outcome-Process Novelty Shaping",
    author = "He, Xu  and
      Guo, Jialiang  and
      Xiong, Fucheng  and
      Zhao, Haodong  and
      li, Xingyang  and
      Zeng, Ke  and
      Cai, Xunliang",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1434/",
    pages = "28721--28739",
    ISBN = "979-8-89176-395-1",
    abstract = "Reinforcement Learning with Verifiable Rewards (RLVR) frequently suffers from mode collapse due to the inherent sparsity of feedback signals. While strategies such as entropy regularization introduce randomness, they lack directionality. Simply incorporating diversity rewards is overly one-sided and fails to identify potential logical errors or hallucinations. To address these limitations, we propose VANE (Value-Aligned Novelty Exploration), a method that simultaneously quantifies novelty across the outcome space (via reward or solution divergence) and the semantic process space (via semantic process divergence). Moreover, VANE employs a value-alignment mechanism that symmetrically amplifies scarce, high-quality solutions while explicitly penalizing diverse yet erroneous reasoning paths. Extensive experiments on models such as Qwen2.5-Math-7B across eight benchmarks{---}encompassing both large-scale mathematical reasoning and out-of-distribution (OOD) tasks{---}demonstrate the effectiveness and generalization of the proposed method."
}