@inproceedings{xie-etal-2026-red,
    title = "Red-Teaming {NSFW} Image Classifiers as Text-to-Image Safeguards",
    author = "Xie, Tinghao  and
      Xie, Yueqi  and
      Zareian, Alireza  and
      Hu, Shuming  and
      Juefei-Xu, Felix  and
      Lin, Xiaowen  and
      Jain, Ankit  and
      Mittal, Prateek  and
      Chen, Li",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/cawl-year/2026.findings-acl.506/",
    doi = "10.18653/v1/2026.findings-acl.506",
    pages = "10413--10441",
    ISBN = "979-8-89176-395-1",
    abstract = "Not Safe for Work (NSFW) image classifiers play a critical role in safeguarding text-to-image (T2I) systems. However, a concerning phenomenon has emerged in T2I systems {--} changes in text prompts that manipulate benign image elements can result in failed detection by NSFW classifiers {--} dubbed ``\textit{context shifts}.'' For instance, while a NSFW image of ``\textit{a nude person in an empty scene}'' can be easily blocked by most NSFW classifiers, a stealthier one that depicts ``\textit{a nude person blending in a group of dressed people}'' may evade detection. We ask: how to systematically reveal NSFW image classifiers' failure against such context shifts?Towards this end, we present an automated red-teaming framework that leverages a set of generative AI tools. We propose an \textbf{exploration-exploitation} approach: \textbf{First}, in the \textit{exploration} stage, we synthesize a diverse and massive 36K NSFW image dataset that facilitates our study of context shifts. We find that varying fractions (e.g., 4.1{\%} to 36{\%} nude and sexual content) of the dataset are misclassified by NSFW image classifiers like GPT-4o and Gemini. \textbf{Second}, in the \textit{exploitation} stage, we leverage these failure cases to train a specialized LLM that rewrites unseen seed prompts into more evasive versions, increasing the likelihood of detection evasion by up to 6 times. Alarmingly, we show \textbf{these failures translate to real-world T2I and even T2V systems} like DALL-E 3, Sora, Nano Banana, and Veo 3 {--} beyond the open-weight image generators in our main study. For example, querying DALL-E 3 with prompts rewritten by our approach increases the chance of obtaining NSFW images from 0 to over 50{\%}."
}