@inproceedings{jiang-etal-2026-coderl,
    title = "{CODERL}+: Improving Code Generation via Reinforcement with Execution Semantics Alignment",
    author = "Jiang, Xue  and
      Dong, Yihong  and
      Liu, Mengyang  and
      Hongyi, Deng  and
      Wang, Tian  and
      Tao, Yongding  and
      Jin, Zhi  and
      Jiao, Wenpin  and
      Li, Ge",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.164/",
    pages = "3608--3622",
    ISBN = "979-8-89176-390-6",
    abstract = "While Large Language Models (LLMs) excel at code generation by learning from vast code corpora, a fundamental semantic gap remains between their training on textual patterns and the goal of functional correctness, which is governed by formal execution semantics. Reinforcement Learning with Verifiable Rewards (RLVR) approaches attempt to bridge this gap using outcome rewards from executing test cases. However, solely relying on binary pass/fail signals is inefficient for establishing a well-aligned connection between the textual representation of code and its execution semantics, especially for subtle logical errors within the code. In this paper, we propose CODERL+, a novel approach that integrates execution semantics alignment into the RLVR training pipeline for code generation. CODERL+ enables the model to infer variable-level execution trajectory, providing a direct learning signal of execution semantics. CODERL+ can construct execution semantics alignment directly using existing on-policy rollouts and integrates seamlessly with various RL algorithms. Extensive experiments demonstrate that CODERL+ outperforms post-training baselines (including RLVR and Distillation), achieving a 4.6{\%} average relative improvement in pass@1. CODERL+ generalizes effectively to other coding tasks, yielding 15.5{\%} and 4.4{\%} higher accuracy on code-reasoning and test-output-generation benchmarks, respectively. CODERL+ shows strong applicability across diverse RL algorithms and LLMs. Furthermore, probe analyses provide compelling evidence that CODERL+ strengthens the alignment between code{'}s textual representations and its underlying execution semantics."
}