Peng Shu
2026
ARGUS: Policy-Adaptive Ad Governance via Evolving Reinforcement with Adversarial Umpiring
Deyi Ji | Junyu Lu | Xuanyi Liu | Liqun Liu | Hailong Zhang | Peng Shu | Huan Yu | Jie Jiang | Tianrun Chen | Lanyun Zhu
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026)
Deyi Ji | Junyu Lu | Xuanyi Liu | Liqun Liu | Hailong Zhang | Peng Shu | Huan Yu | Jie Jiang | Tianrun Chen | Lanyun Zhu
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026)
Online advertising governance faces significant challenges due to the non-stationary nature of regulatory policies, where emerging mandates (e.g., restrictions on education or aesthetic anxiety) create severe label inconsistencies and reasoning ambiguities in historical datasets. In this paper, we propose ARGUS, a policy-adaptive governance system that enables evolving reinforcement through multi-agent adversarial umpiring. ARGUS addresses the sparsity of new policy data by employing a three-stage framework: (1) Policy Seeding for initial perception; (2) Adversarial Label Rectification, which utilizes a ”Prosecutor-Defender-Umpire” architecture to resolve conflicts between stale labels and new mandates; and (3) Latent Knowledge Discovery, which employs a tripartite dialectical discussion to unearth sophisticated, “gray-area” violations. By leveraging RAG-enhanced policy knowledge and Chain-of-Thought synthesis as dynamic rewards for reinforcement learning, ARGUS synchronizes its reasoning pathways with evolving regulations. Extensive experiments on both industrial and public datasets demonstrate that ARGUS significantly outperforms traditional fine-tuning baselines, achieving superior policy-adaptive learning with minimal gold data.
Towards Faithful Industrial RAG: A Reinforced Co-adaptation Framework for Advertising QA
Wenwei Li | Ming Xu | Tianle Xia | Lingxiang Hu | Yiding Sun | Linfang Shang | Liqun Liu | Peng Shu | Huan Yu | Jie Jiang
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026)
Wenwei Li | Ming Xu | Tianle Xia | Lingxiang Hu | Yiding Sun | Linfang Shang | Liqun Liu | Peng Shu | Huan Yu | Jie Jiang
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026)
Industrial advertising question answering (QA) is a high-stakes task in which hallucinated content, particularly fabricated URLs, can lead to financial loss, compliance violations, and legal risk. Although Retrieval-Augmented Generation (RAG) is widely adopted, deploying it in production remains challenging because industrial knowledge is inherently relational, frequently updated, and insufficiently aligned with generation objectives. We propose a reinforced co-adaptation framework that jointly optimizes retrieval and generation through two components: (1) Graph-aware Retrieval (GraphRAG), which models entity-relation structure over a high-citation knowledge subgraph for multi-hop, domain-specific evidence selection; and (2) evidence-constrained reinforcement learning via Group Relative Policy Optimization (GRPO) with multi-dimensional rewards covering faithfulness, style compliance, safety, and URL validity. Experiments on an internal advertising QA dataset show consistent gains across expert-judged dimensions including accuracy, completeness, and safety, while reducing the hallucination rate by 72%. A two-week online A/B test demonstrates a 28.6% increase in like rate, a 46.2% decrease in dislike rate, and a 92.7% reduction in URL hallucination. The system has been running in production for over half a year and has served millions of QA interactions.
Search-P1: Path-Centric Reward Shaping for Stable and Efficient Agentic RAG Training
Tianle Xia | Ming Xu | Lingxiang Hu | Yiding Sun | Wenwei Li | Linfang Shang | Liqun Liu | Peng Shu | Huan Yu | Jie Jiang
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026)
Tianle Xia | Ming Xu | Lingxiang Hu | Yiding Sun | Wenwei Li | Linfang Shang | Liqun Liu | Peng Shu | Huan Yu | Jie Jiang
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026)
Retrieval-Augmented Generation (RAG) enhances large language models (LLMs) by incorporating external knowledge, yet traditional single-round retrieval struggles with complex multi-step reasoning.Agentic RAG addresses this by enabling LLMs to dynamically decide when and what to retrieve, but current RL-based training methods suffer from sparse outcome rewards that discard intermediate signals and low sample efficiency where failed samples contribute nothing.We propose Search-P1, a framework that introduces path-centric reward shaping for agentic RAG training, comprising two key components: (1) Path-Centric Reward, which evaluates the structural quality of reasoning trajectories through order-agnostic step coverage and soft scoring that extracts learning signals even from failed samples, and (2) Dual-Track Path Scoring with offline-generated reference planners that assesses paths from both self-consistency and reference-alignment perspectives.Experiments on multiple QA benchmarks demonstrate that Search-P1 achieves significant improvements over Search-R1 and other strong baselines, with an average accuracy gain of 7.7 points.
ℛ3: Advertisement Compliance ℛectification via Group-ℛelative Experience Extractor and Curriculum ℛeinforcement
Yuan Chen | Zhenyu Hu | Mengge Xue | Cao Te | Liqun Liu | Peng Shu | Huan Yu | Jie Jiang
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026)
Yuan Chen | Zhenyu Hu | Mengge Xue | Cao Te | Liqun Liu | Peng Shu | Huan Yu | Jie Jiang
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026)
Rigorous content moderation is crucial for online advertising but leads to millions of daily rejections. This scale renders manual rectification infeasible, particularly for video advertisements.However, existing safety-driven methods often suffer from aggressive over-editing, which compromises the advertiser’s original semantic intent merely to satisfy compliance.In this work, we target the rectification of textual violations in video ads, covering both speech transcripts and on-screen text. We propose ℛ3, a novel framework designed to harmonize compliance with original semantic intent preservation.Our approach integrates three key innovations: (1) an experience-driven data synthesis framework that bootstraps high-quality supervision via group-**R**elative compliance experience extractor; (2) a curriculum **R**einforcement learning strategy with hierarchical rewards designed to enforce compliance while maximizing semantic consistency;and (3) a comprehensive video **R**ectification framework seamlessly integrating text recognition, rewriting, and re-rendering for industrial deployment. Extensive experiments on industrial datasets and online A/B testing demonstrate that ℛ3 significantly outperforms state-of-the-art baselines, achieving an optimal trade-off between violation rectification and intent preservation.
SSR-A: Spatial- and Semantic-Aware Instructions and Curriculum Reinforcement for Advertisement Compliant Rectification
Cao Te | Mengge Xue | Zhenyu Hu | Yuan Chen | Liqun Liu | Peng Shu | Huan Yu | Jie Jiang
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026)
Cao Te | Mengge Xue | Zhenyu Hu | Yuan Chen | Liqun Liu | Peng Shu | Huan Yu | Jie Jiang
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026)
While advertising is a cornerstone of commercial growth, it is constrained by online violation detection systems that reject non-compliant content at a million-scale daily. Advertisers urgently require automated solutions to rectify these advertisements, especially visual ads, as manual fixing is unscalable. Although recent safety-driven methods can achieve compliance, they typically suffer from over-editing, destroying the original commercial intent and perceptual similarity.To address this, we present SSR-A, a framework tailored for the minimalist rectification of non-compliant image ads.Instead of fine-tuning image editing models directly, SSR-A focuses on translating violation policies into targeted editing instructions.We first introduce a Spatial- and Semantic-Aware Instruction Synthesis Pipeline, where MLLMs synthesize candidate instructions—incorporating spatial grounding and semantic guidance—and select the optimal instruction via multi-dimensional evaluation. Furthermore, we align the model using Curriculum Reinforcement Learning, employing GRPO with multi-faceted rewards to progressively navigate the trade-off between compliance and visual preservation. Extensive experiments and online A/B tests show that SSR-A significantly outperforms state-of-the-art baselines in both compliance and preservation of visual and commercial consistency.
2025
Real-time Ad Retrieval via LLM-generative Commercial Intention for Sponsored Search Advertising
Tongtong Liu | Zhaohui Wang | Meiyue Qin | Zenghui Lu | Xudong Chen | Yuekui Yang | Peng Shu
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Tongtong Liu | Zhaohui Wang | Meiyue Qin | Zenghui Lu | Xudong Chen | Yuekui Yang | Peng Shu
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
The integration of Large Language Models (LLMs) with retrieval systems has shown promising potential in retrieving documents (docs) or advertisements (ads) for a given query. Existing LLM-based retrieval methods generate numeric or content-based DocIDs to retrieve docs/ads. However, the one-to-few mapping between numeric IDs and docs, along with the time-consuming content extraction, leads to semantic inefficiency and limits the scalability of existing methods on large-scale corpora. In this paper, we propose the **R**eal-time **A**d **RE**trieval (RARE) framework, which leverages LLM-generated text called Commercial Intentions (CIs) as an intermediate semantic representation to directly retrieve ads for queries in real-time. These CIs are generated by a customized LLM injected with commercial knowledge, enhancing its domain relevance. Each CI corresponds to multiple ads, yielding a lightweight and scalable set of CIs. RARE has been implemented in a real-world online system, handling daily search volumes in billions. The online implementation has yielded significant benefits: a 5.04% increase in consumption, a 6.37% rise in Gross Merchandise Volume (GMV), a 1.28% enhancement in click-through rate (CTR) and a 5.29% increase in shallow conversions. Extensive offline experiments show RARE’s superiority over ten competitive baselines in four major categories.
RAVEN++: Pinpointing Fine-Grained Violations in Advertisement Videos with Active Reinforcement Reasoning
Deyi Ji | Yuekui Yang | Liqun Liu | Peng Shu | Haiyang Wu | Shaogang Tang | Xudong Chen | Shaoping Ma | Tianrun Chen | Lanyun Zhu
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track
Deyi Ji | Yuekui Yang | Liqun Liu | Peng Shu | Haiyang Wu | Shaogang Tang | Xudong Chen | Shaoping Ma | Tianrun Chen | Lanyun Zhu
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track
Advertising (Ad) is a cornerstone of the digital economy, yet the moderation of video advertisements remains a significant challenge due to their complexity and the need for precise violation localization. While recent advancements, such as the RAVEN model, have improved coarse-grained violation detection, critical gaps persist in fine-grained understanding, explainability, and generalization. To address these limitations, we propose RAVEN++, a novel framework that introduces three key innovations: 1) Active Reinforcement Learning (RL), which dynamically adapts training to samples of varying difficulty; 2) Fine-Grained Violation Understanding, achieved through hierarchical reward functions and reasoning distillation; and 3) Progressive Multi-Stage Training, which systematically combines knowledge injection, curriculum-based passive RL, and active RL. Extensive experiments on both public and proprietary datasets, on both offline scenarios and online deployed A/B Testing, demonstrate that RAVEN++ outperforms general-purpose LLMs and specialized models like RAVEN in terms of fine-grained violation understanding, reasoning capabilities, and generalization ability.