@inproceedings{roll-etal-2026-text,
    title = "The Text Aphasia Battery ({TAB}): A Clinically-Grounded Benchmark for Aphasia-Like Deficits in Language Models",
    author = "Roll, Nathan  and
      Kries, Jill  and
      Jin, Flora  and
      Wang, Catherine  and
      Finley, Ann Marie  and
      Sumner, Meghan  and
      Shain, Cory  and
      Gwilliams, Laura",
    editor = "Zirikly, Aya  and
      Bar, Kfir  and
      MacAvaney, Sean  and
      Ireland, Molly  and
      Ophir, Yaakov  and
      Atzil-Slonim, Dana  and
      Varadarajan, Vasudha  and
      Bedrick, Steven  and
      Desmet, Bart",
    booktitle = "Proceedings of the 10th Workshop on Computational Linguistics and Clinical Psychology ({CLP}sych 2026)",
    month = jul,
    year = "2026",
    address = "San Diego, California, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl-workshops/2026.clpsych-1.27/",
    pages = "340--354",
    ISBN = "979-8-89176-421-7",
    abstract = "Large language models (LLMs) have emerged as a candidate `model organism' for human language, offering an unprecedented opportunity to study the computational basis of linguistic disorders like aphasia. However, traditional clinical assessments are ill-suited for LLMs, as they presuppose human-like pragmatic pressures and probe cognitive processes not inherent to artificial architectures. We introduce the Text Aphasia Battery (TAB), a text-only benchmark adapted from the Quick Aphasia Battery (QAB) to assess aphasic-like deficits in LLMs. The TAB comprises four subtests: Connected Text, Word Comprehension, Sentence Comprehension, and Repetition. This paper details the TAB{'}s design, subtests, and scoring criteria. To facilitate large-scale use, we validate an automated evaluation protocol using Gemini 2.5 Flash, which achieves reliability comparable to expert human raters (prevalence-weighted Cohen{'}s k=0.255 for model{--}consensus agreement vs. 0.286 for human{--}human agreement). We release TAB as a clinically-grounded, scalable framework for analyzing language deficits in artificial systems."
}