@inproceedings{cai-etal-2026-codecontests,
    title = "{C}ode{C}ontests-{O}: Powering {LLM}s via Feedback-Driven Iterative Test Case Generation",
    author = "Cai, Jianfeng  and
      Zhu, Jinhua  and
      Sun, Ruopei  and
      Zhao, Kangwen  and
      Xue, Dongyun  and
      Feng, Mingxiao  and
      Zhou, Wengang  and
      Li, Houqiang",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.53/",
    pages = "1054--1072",
    ISBN = "979-8-89176-395-1",
    abstract = "The rise of reasoning models necessitates large-scale verifiable data, for which programming tasks serve as an ideal source. However, while competitive programming platforms provide abundant problems and solutions, high-quality test cases for verification remain scarce. Existing approaches attempt to synthesize test cases using Large Language Models (LLMs), but rely solely on the model{'}s intrinsic generation capabilities without external feedback, frequently resulting in insufficiently diverse cases. To address this limitation, we propose a $\textbf{Feedback-Driven Iterative Framework}$ for comprehensive test case construction. Specifically, our method leverages the LLM to generate initial test cases, executes them against known correct and incorrect solutions, and utilizes the failed results as feedback to guide the LLM in refining the test cases toward high fidelity and discriminability. We then apply this method to the CodeContests dataset to construct an optimized high-quality derivative, $\textbf{CodeContests-O}$. Evaluating against the entire pool of solutions ($1.1 \times 10^7$ in total), our dataset achieves an average True Positive Rate (TPR) of 89.35{\%} and True Negative Rate (TNR) of 90.30{\%}, significantly outperforming the CodeContests and CodeContests+ by margins of 4.30{\%} and 8.78{\%}, respectively. Furthermore, fine-tuning the Qwen2.5-7B model on CodeContests-O results in a 9.52{\%} improvement on LiveCodeBench (Pass@1). Experiments demonstrate the effectiveness of our framework and the quality of CodeContests-O."
}