@inproceedings{oh-etal-2026-subject,
    title = "Subject-level Inference for Realistic Text Anonymization Evaluation",
    author = "Oh, Myeong Seok  and
      Kim, Dong-Yun  and
      Oh, Hanseok  and
      Kang, Chaean  and
      Kang, Joeun  and
      Wang, Xiaonan  and
      Park, Hyunjung  and
      Jung, Young Cheol  and
      Kim, Hansaem",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.778/",
    pages = "17100--17135",
    ISBN = "979-8-89176-390-6",
    abstract = "Current text anonymization evaluation relies on span-based metrics that fail to capture what an adversary could actually infer, and assumes a single data subject, ignoring multi-subject scenarios. To address these limitations, we present SPIA (Subject-level PII Inference Assessment), the first benchmark that shifts the unit of evaluation from text spans to individuals, comprising 675 documents across legal and online domains with novel subject-level protection metrics. Extensive experiments show that even when over 90{\%} of PII spans are masked, subject-level inference protection drops as low as 33{\%}, leaving the majority of personal information recoverable through contextual inference. Furthermore, target-subject-focused anonymization leaves non-target subjects substantially more exposed than the target subject. We show that subject-level inference-based evaluation is essential for ensuring safe text anonymization in real-world settings."
}