@inproceedings{hao-etal-2026-veg,
    title = "{VEG}: Verbal $\epsilon$-greedy for Semantic Exploration in Multi-Turn {RL} Agents",
    author = "Hao, Yongchang  and
      Hao, Jie  and
      Mei, Yongsheng  and
      Ye, Ze  and
      Chai, Junyi  and
      Guo, Bin  and
      Yao, Benjamin Z.  and
      Guo, Chenlei  and
      Mou, Lili",
    editor = "Li, Yunyao  and
      Rehm, Georg  and
      Tu, Mei",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics ({ACL} 2026)",
    month = jul,
    year = "2026",
    address = "San Diego, California, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-industry.82/",
    pages = "1159--1169",
    ISBN = "979-8-89176-394-4",
    abstract = "Reinforcement learning (RL) has become a cornerstone of the post-training pipeline for large language models (LLMs), enabling capabilities such as complex reasoning and tool use. However, standard RL approaches face significant challenges due to reward sparsity. Moreover, LLMs typically exhibit mode-seeking behavior, concentrating probability mass on high-likelihood regions. This lack of diversity biases the model toward premature exploitation, hindering the exploration necessary for optimal learning. To address this, we propose VEG (verbal $\epsilon$-greedy), a novel framework that leverages external feedback as a dynamic control variable to explicitly balance exploration and exploitation within the semantic space. This method not only supplements sparse final rewards with intermediate signals but also enforces sustained exploration throughout the training process. Experiments on Tau Bench and SearchQA demonstrate that our method achieves superior accuracy compared to standard RL baselines. Notably, the trained policy eventually outperforms the external feedback model itself, demonstrating that VEG enables the agent to effectively filter and improve upon the guidance it receives."
}