Partners in Crime: Multi-view Sequential Inference for Movie Understanding

Nikos Papasarantopoulos, Lea Frermann, Mirella Lapata, Shay B. Cohen

Abstract
Multi-view learning algorithms are powerful representation learning tools, often exploited in the context of multimodal problems. However, for problems requiring inference at the token-level of a sequence (that is, a separate prediction must be made for every time step), it is often the case that single-view systems are used, or that more than one views are fused in a simple manner. We describe an incremental neural architecture paired with a novel training objective for incremental inference. The network operates on multi-view data. We demonstrate the effectiveness of our approach on the problem of predicting perpetrators in crime drama series, for which our model significantly outperforms previous work and strong baselines. Moreover, we introduce two tasks, crime case and speaker type tagging, that contribute to movie understanding and demonstrate the effectiveness of our model on them.
Anthology ID:
D19-1212
Volume:
Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
Month:
November
Year:
2019
Address:
Hong Kong, China
Venues:
EMNLP | IJCNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
2057–2067
Language:
URL:
https://aclanthology.org/D19-1212
DOI:
10.18653/v1/D19-1212
Bibkey:
Cite (ACL):
Nikos Papasarantopoulos, Lea Frermann, Mirella Lapata, and Shay B. Cohen. 2019. Partners in Crime: Multi-view Sequential Inference for Movie Understanding. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2057–2067, Hong Kong, China. Association for Computational Linguistics.
Cite (Informal):
Partners in Crime: Multi-view Sequential Inference for Movie Understanding (Papasarantopoulos et al., EMNLP-IJCNLP 2019)
Copy Citation:
PDF:
https://preview.aclanthology.org/auto-file-uploads/D19-1212.pdf
PDF Search