@inproceedings{guo-etal-2026-rethinking-multiple,
    title = "Rethinking Multiple-Choice Questions for {RLVR}: Unlocking Potential via Distractor Design",
    author = "Guo, Xu  and
      Ge, Qiming  and
      Tong, Jian  and
      Chen, Kedi  and
      Zhang, Jin  and
      Yang, Xiaogui  and
      Gao, Xuan  and
      Lv, Haijun  and
      Lu, Zhihui  and
      Zou, Yicheng  and
      Guo, Qipeng",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1003/",
    pages = "20092--20113",
    ISBN = "979-8-89176-395-1",
    abstract = "Reinforcement Learning with Verifiable Rewards (RLVR) significantly enhances the reasoning capabilities of Large Language Models. When applied to RLVR, Multiple-Choice Questions (MCQs) offer a scalable source of verifiable data but risk inducing reward hacking, where models shortcut reasoning via random guessing or simple elimination. Current approaches often mitigate this by converting MCQs to open-ended formats, thereby discarding the contrastive signal provided by expert-designed distractors. In this work, we systematically investigate the impact of option design on RLVR. Our analysis highlights two primary insights: (1) Mismatches in option counts between training and testing degrade performance. (2) Strong distractors effectively mitigate random guessing, enabling effective RLVR training even with 2-way questions. Motivated by these findings, we propose Iterative Distractor Curation (IDC), a framework that actively constructs high-quality distractors to block elimination shortcuts and promote deep reasoning. Experiments on various benchmarks demonstrate that our method effectively enhances distractor quality and yields significant gains in RLVR training compared to the original data."
}