@inproceedings{liu-etal-2025-counterfactual,
    title = "Counterfactual Evaluation for Blind Attack Detection in {LLM}-based Evaluation Systems",
    author = "Liu, Lijia  and
      Kondo, Takumi  and
      Atarashi, Kyohei  and
      Takeuchi, Koh  and
      Li, Jiyi  and
      Saito, Shigeru  and
      Kashima, Hisashi",
    editor = "Inui, Kentaro  and
      Sakti, Sakriani  and
      Wang, Haofen  and
      Wong, Derek F.  and
      Bhattacharyya, Pushpak  and
      Banerjee, Biplab  and
      Ekbal, Asif  and
      Chakraborty, Tanmoy  and
      Singh, Dhirendra Pratap",
    booktitle = "Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics",
    month = dec,
    year = "2025",
    address = "Mumbai, India",
    publisher = "The Asian Federation of Natural Language Processing and The Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-ijcnlp-aacl/2025.ijcnlp-long.33/",
    pages = "572--584",
    ISBN = "979-8-89176-298-5",
    abstract = "This paper investigates defenses in LLM-based evaluation, where prompt injection attacks can manipulate scores by deceiving the evaluation system. We formalize blind attacks as a class in which candidate answers are crafted independently of the true answer. To counter such attacks, we propose an evaluation framework that combines standard and counterfactual evaluation. Experiments show it significantly improves attack detection with minimal performance trade-offs for recent models."
}