@inproceedings{zeng-etal-2026-teaching,
    title = "Teaching {LLM} to be Persuasive: Reward-Enhanced Policy Optimization for Alignment from Heterogeneous Rewards",
    author = "Zeng, Xia  and
      Chen, Yihan  and
      Liu, Luhui  and
      Luo, Chao  and
      Chen, Ye  and
      Zhuangzhuoran",
    editor = "Li, Yunyao  and
      Rehm, Georg  and
      Tu, Mei",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics ({ACL} 2026)",
    month = jul,
    year = "2026",
    address = "San Diego, California, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-industry.34/",
    pages = "494--506",
    ISBN = "979-8-89176-394-4",
    abstract = "We deploy large language models (LLMs) as business development (BD) agents for persuasive price negotiation in online travel agencies (OTAs). The agent must follow a multi-stage Standard Operating Procedure (SOP) and strict guardrails (no over-promising and no hallucinations), while remaining human-like and effective over long, multi-turn dialogues.We propose Reward-Enhanced Policy Optimization (REPO), a reinforcement learning post-training method that combines heterogeneous rewards: a preference-trained reward model (RM), an LLM-as-a-judge (RJ) for nuanced behaviors (e.g., emotional value and SOP compliance), and rule-based reward functions (RF) (mainly regex-based) for deterministic checks on numerics, formatting, and guardrails. In expert consensus evaluation (three human experts; 30 online conversations and 45 curated bad cases), REPO improves average dialogue rating to 4.63 (+0.33 over GRPO) and raises the share of conversations with at least one excellent response to 66.67{\%} (+23.34 pp over GRPO), while achieving a 93.33{\%} bad-case fix rate with 75.56{\%} clean fixes.In a production A/B test on 9{,}653 real customer conversations (vs. an intent-driven dialogue system), REPO improves response rate by +12.14 pp and task success rate by +5.94 pp ($p<0.001$)."
}