Chop and Change: Anaphora Resolution in Instructional Cooking Videos
Cennet Oguz, Ivana Kruijff-Korbayova, Emmanuel Vincent, Pascal Denis, Josef van Genabith
Abstract
Linguistic ambiguities arising from changes in entities in action flows are a key challenge in instructional cooking videos. In particular, temporally evolving entities present rich and to date understudied challenges for anaphora resolution. For example “oil” mixed with “salt” is later referred to as a “mixture”. In this paper we propose novel annotation guidelines to annotate recipes for the anaphora resolution task, reflecting change in entities. Moreover, we present experimental results for end-to-end multimodal anaphora resolution with the new annotation scheme and propose the use of temporal features for performance improvement.- Anthology ID:
- 2022.findings-aacl.34
- Volume:
- Findings of the Association for Computational Linguistics: AACL-IJCNLP 2022
- Month:
- November
- Year:
- 2022
- Address:
- Online only
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 364–374
- Language:
- URL:
- https://aclanthology.org/2022.findings-aacl.34
- DOI:
- Cite (ACL):
- Cennet Oguz, Ivana Kruijff-Korbayova, Emmanuel Vincent, Pascal Denis, and Josef van Genabith. 2022. Chop and Change: Anaphora Resolution in Instructional Cooking Videos. In Findings of the Association for Computational Linguistics: AACL-IJCNLP 2022, pages 364–374, Online only. Association for Computational Linguistics.
- Cite (Informal):
- Chop and Change: Anaphora Resolution in Instructional Cooking Videos (Oguz et al., Findings 2022)
- PDF:
- https://preview.aclanthology.org/auto-file-uploads/2022.findings-aacl.34.pdf