Abstract
Modeling thematic fit (a verb-argument compositional semantics task) currently requires a very large burden of labeled data. We take a linguistically machine-annotated large corpus and replace corpus layers with output from higher-quality, more modern taggers. We compare the old and new corpus versions’ impact on a verb-argument fit modeling task, using a high-performing neural approach. We discover that higher annotation quality dramatically reduces our data requirement while demonstrating better supervised predicate-argument classification. But in applying the model to psycholinguistic tasks outside the training objective, we see clear gains at scale, but only in one of two thematic fit estimation tasks, and no clear gains on the other. We also see that quality improves with training size, but perhaps plateauing or even declining in one task. Last, we tested the effect of role set size. All this suggests that the quality/quantity interplay is not all you need. We replicate previous studies while modifying certain role representation details and set a new state-of-the-art in event modeling, using a fraction of the data. We make the new corpus version public.- Anthology ID:
- 2022.lrec-1.556
- Volume:
- Proceedings of the Thirteenth Language Resources and Evaluation Conference
- Month:
- June
- Year:
- 2022
- Address:
- Marseille, France
- Editors:
- Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Jan Odijk, Stelios Piperidis
- Venue:
- LREC
- SIG:
- Publisher:
- European Language Resources Association
- Note:
- Pages:
- 5188–5197
- Language:
- URL:
- https://aclanthology.org/2022.lrec-1.556
- DOI:
- Cite (ACL):
- Yuval Marton and Asad Sayeed. 2022. Thematic Fit Bits: Annotation Quality and Quantity Interplay for Event Participant Representation. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 5188–5197, Marseille, France. European Language Resources Association.
- Cite (Informal):
- Thematic Fit Bits: Annotation Quality and Quantity Interplay for Event Participant Representation (Marton & Sayeed, LREC 2022)
- PDF:
- https://preview.aclanthology.org/naacl-24-ws-corrections/2022.lrec-1.556.pdf
- Data
- FrameNet, Penn Treebank