@inproceedings{tam-etal-2025-none,
    title = "None of the Above, Less of the Right Parallel Patterns in Human and {LLM} Performance on Multi-Choice Questions Answering",
    author = "Tam, Zhi Rui  and
      Wu, Cheng-Kuang  and
      Lin, Chieh-Yen  and
      Chen, Yun-Nung",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingestion-acl-25/2025.findings-acl.1031/",
    pages = "20112--20134",
    ISBN = "979-8-89176-256-5",
    abstract = "Multiple-choice exam questions with ``None of the above'' (NA) options have been extensively studied in educational testing, in which existing research suggests that they better assess true knowledge. However, their impact on Large Language Models (LLMs) evaluation remains underexplored. Through systematic experiments with 28 LLMs on the MMLU benchmark, we examine how NA options affect model performance and confidence calibration. Our analysis reveals that NA options, when used as the correct answer, lead to a consistent 30-50{\%} performance drop across models regardless of scale{--}suggesting that LLMs lack the meta-cognitive ability to systematically evaluate and reject all given options when none are correct. This degradation shows strong domain dependence, with minimal impact on mathematical reasoning (14.6{\%} drop) but severe effects on tasks requiring uncertainty handling like business ethics (48.1{\%} drop). Our results highlight important implications for benchmark design and raise questions about LLMs' ability to handle uncertainty in real-world applications."
}