@inproceedings{hamilton-etal-2026-narrabench,
    title = "{N}arra{B}ench: A Comprehensive Framework for Narrative Benchmarking",
    author = "Hamilton, Sil  and
      Wilkens, Matthew  and
      Piper, Andrew",
    editor = "Demberg, Vera  and
      Inui, Kentaro  and
      Marquez, Llu{\'i}s",
    booktitle = "Proceedings of the 19th Conference of the {E}uropean Chapter of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = mar,
    year = "2026",
    address = "Rabat, Morocco",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-eacl/2026.eacl-long.176/",
    pages = "3786--3801",
    ISBN = "979-8-89176-380-7",
    abstract = "We present NarraBench, a theory-informed taxonomy of narrative-understanding tasks, as well as an associated survey of 78 existing benchmarks in the area. We find significant need for new evaluations covering aspects of narrative understanding that are either overlooked in current work or are poorly aligned with existing metrics. Specifically, we estimate that only 27{\%} of narrative tasks are well captured by existing benchmarks, and we note that some areas {--} including narrative events, style, perspective, and revelation {--} are nearly absent from current evaluations. We also note the need for increased development of benchmarks capable of assessing constitutively subjective and perspectival aspects of narrative, that is, aspects for which there is generally no single correct answer. Our taxonomy, survey, and methodology are of value to NLP researchers seeking to test LLM narrative understanding."
}