@inproceedings{song-etal-2025-addriftbench,
    title = "{A}d{D}rift{B}ench: A Benchmark for Detecting Data Drift and Label Drift in Short Video Advertising",
    author = "Song, Yinghao  and
      Zeng, Xiangji  and
      Cui, Shuai  and
      Sun, Lu  and
      Liu, Zhaowei  and
      Yuan, Yuan  and
      Wang, Yulu  and
      Zhou, Hai  and
      Gong, Zhaohan",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2025",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/author-page-yu-wang-polytechnic/2025.findings-emnlp.545/",
    doi = "10.18653/v1/2025.findings-emnlp.545",
    pages = "10305--10321",
    ISBN = "979-8-89176-335-7",
    abstract = "With the commercialization of short video platforms (SVPs), the demand for compliance auditing of advertising content has grown rapidly. The rise of large vision-language models (VLMs) offers new opportunities for automating ad content moderation. However, short video advertising scenarios present unique challenges due to $\textbf{data drift (DD)}$ and $\textbf{label drift (LD)}$. DD refers to rapid shifts in data distribution caused by advertisers to evade platform review mechanisms. LD arises from the evolving and increasingly standardized review guidelines of SVPs, which effectively alter the classification boundaries over time. Despite the significance of these phenomena, there is currently a lack of benchmark tools designed to evaluate model performance under such conditions. To address this gap, we propose $\textbf{AdDriftBench (ADB)}$. The ADB dataset consists of 3,480 short video ads, including 2,280 examples labeled under data drift scenarios, designed to evaluate the generalization capabilities of VLMs under rapidly shifting content distributions. An additional 1,200 examples represent label drift scenarios, aimed at assessing VLMs' abilities in instruction following and fine-grained semantic understanding under varying auditing standards. Through extensive experiments on 16 open-source VLMs, we find that current models perform moderately in short video advertising contexts, particularly in handling fine-grained semantics and adapting to shifting instructions. Our dataset will be made publicly available."
}