A Qualia-Based Audit of Procedural Event Annotations

Kyeongmin Rim, Marc Verhagen, James Pustejovsky


Abstract
Procedural event annotations record *what changed* but not the semantic relevance or grounding of the change: whether the annotated entity is the kind of thing whose state matters for the domain.We present Entity Qualia Structure (EQS), a per-entity sortal-type categorization (coarsened from Generative Lexicon’s type system to three categories: natural, artifactual, instrument) extracted from existing lexical resources.Applied to the OpenPI food domain, EQS reaches 84.7% coverage of the 518-item entity vocabulary; across 9367 transformation annotations, only 51.1% concern food entities themselves, while 30.2% record state changes of instruments, entities whose sortal type places them outside the food-state task.In a three-way comparison against existing cleanup efforts, EQS uniquely flags 15.6% of annotations that neither human re-annotation (OpenPI-C) nor LLM salience scoring (OpenPI 2.0) catches.Analysis of the *agentive* quale reveals that 93% of agentive-positive annotations involve instruments rather than food: entity creation can only be detected when the agentive feature is paired with the associated verb’s event semantics.
Anthology ID:
2026.eeuca-1.6
Volume:
Proceedings of the 9th Workshop on Event Extraction and Understanding: Challenges and Applications (EEUCA 2026)
Month:
July
Year:
2026
Address:
San Diego, California, USA
Editors:
Ali Hürriyetoğlu, Surendrabikram Thapa, Hristo Tanev
Venues:
EEUCA | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
49–57
Language:
URL:
https://preview.aclanthology.org/ingest-acl-workshops/2026.eeuca-1.6/
DOI:
Bibkey:
Cite (ACL):
Kyeongmin Rim, Marc Verhagen, and James Pustejovsky. 2026. A Qualia-Based Audit of Procedural Event Annotations. In Proceedings of the 9th Workshop on Event Extraction and Understanding: Challenges and Applications (EEUCA 2026), pages 49–57, San Diego, California, USA. Association for Computational Linguistics.
Cite (Informal):
A Qualia-Based Audit of Procedural Event Annotations (Rim et al., EEUCA 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl-workshops/2026.eeuca-1.6.pdf