The Aligned Multimodal Movie Treebank: An audio, video, dependency-parse treebank

Adam Yaari, Jan DeWitt, Henry Hu, Bennett Stankovits, Sue Felshin, Yevgeni Berzak, Helena Aparicio, Boris Katz, Ignacio Cases, Andrei Barbu

Abstract
Treebanks have traditionally included only text and were derived from written sources such as newspapers or the web. We introduce the Aligned Multimodal Movie Treebank (AMMT), an English language treebank derived from dialog in Hollywood movies which includes transcriptions of the audio-visual streams with word-level alignment, as well as part of speech tags and dependency parses in the Universal Dependencies formalism. AMMT consists of 31,264 sentences and 218,090 words, that will amount to the 3rd largest UD English treebank and the only multimodal treebank in UD. To help with the web-based annotation effort, we also introduce the Efficient Audio Alignment Annotator (EAAA), a companion tool that enables annotators to significantly speed-up their annotation processes.
Anthology ID:
2022.emnlp-main.648
Volume:
Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
Month:
December
Year:
2022
Address:
Abu Dhabi, United Arab Emirates
Editors:
Yoav Goldberg, Zornitsa Kozareva, Yue Zhang
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
9531–9539
Language:
URL:
https://aclanthology.org/2022.emnlp-main.648
DOI:
10.18653/v1/2022.emnlp-main.648
Bibkey:
Cite (ACL):
Adam Yaari, Jan DeWitt, Henry Hu, Bennett Stankovits, Sue Felshin, Yevgeni Berzak, Helena Aparicio, Boris Katz, Ignacio Cases, and Andrei Barbu. 2022. The Aligned Multimodal Movie Treebank: An audio, video, dependency-parse treebank. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 9531–9539, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Cite (Informal):
The Aligned Multimodal Movie Treebank: An audio, video, dependency-parse treebank (Yaari et al., EMNLP 2022)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-2/2022.emnlp-main.648.pdf
PDF Search