@inproceedings{fane-etal-2025-bemeae,
    title = "{BEMEAE}: Moving Beyond Exact Span Match for Event Argument Extraction",
    author = "Fane, Enfa  and
      Uddin, Md Nayem  and
      Ikumariegbe, Oghenevovwe  and
      Kashif, Daniyal  and
      Blanco, Eduardo  and
      Corman, Steven",
    editor = "Chiruzzo, Luis  and
      Ritter, Alan  and
      Wang, Lu",
    booktitle = "Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)",
    month = apr,
    year = "2025",
    address = "Albuquerque, New Mexico",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/moar-dois/2025.naacl-long.295/",
    doi = "10.18653/v1/2025.naacl-long.295",
    pages = "5734--5749",
    ISBN = "979-8-89176-189-6",
    abstract = "Event Argument Extraction (EAE) is a key task in natural language processing, focusing on identifying and classifying event arguments in text. However, the widely adopted exact span match (ESM) evaluation metric has notable limitations due to its rigid span constraints, often misidentifying valid predictions as errors and underestimating system performance. In this paper, we evaluate nine state-of-the-art EAE models on the RAMS and GENEVA datasets, highlighting ESM{'}s limitations. To address these issues, we introduce BEMEAE (Beyond Exact Span Match for Event Argument Extraction), a novel evaluation metric that recognizes predictions that are semantically equivalent to or improve upon the reference. BEMEAE integrates deterministic components with a semantic matching component for more accurate assessment. Our experiments demonstrate that BEMEAE aligns more closely with human judgments. We show that BEMEAE not only leads to higher F1 scores compared to ESM but also results in significant changes in model rankings, underscoring ESM{'}s inadequacy for comprehensive evaluation of EAE."
}