@inproceedings{chen-etal-2026-aipo,
    title = "{AIPO}: Adaptive Information Guided Token-Level Reinforcement Learning for Large Language Model Reasoning",
    author = "Chen, Bin  and
      Ye, Hongfei  and
      Wang, Huiyang  and
      Liu, Wenxi  and
      Zhang, Yu  and
      Liu, Furui",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.2057/",
    pages = "44441--44450",
    ISBN = "979-8-89176-390-6",
    abstract = "Reinforcement Learning with Verifiable Rewards (RLVR) improves the reasoning capability of Large Language Models (LLMs). Current RLVR trains LLMs on all generated tokens, rather than exploring which tokens actually contribute to reasoning. We propose AIPO(Adaptive{--}Information Policy Optimization), which focuses updates on those decisive tokens discovered on the fly. AIPO estimates each hidden state{'}s mutual information to score tokens. Policy gradients are then computed only on these critical tokens, using an advantage that blends information gain and verifiable correctness. To improve the efficiency of mutual-information estimation, AIPO adopts a Random{--}Fourier approximation of the Hilbert{--}Schmidt Independence Criterion. Across five math and science benchmarks, AIPO yields up to +20{\%} accuracy over strong RLVR baselines while updating merely 10{\%} of tokens, demonstrating superior efficiency and effectiveness. Our findings highlight the importance of information{--}driven token selection for efficient and effective reinforcement learning of LLM reasoning."
}