VKIE: The Application of Key Information Extraction on Video Text

Siyu An, Ye Liu, Haoyuan Peng, Di Yin


Abstract
Extracting structured information from videos is critical for numerous downstream applications in the industry. In this paper, we define a significant task of extracting hierarchical key information from visual texts on videos. To fulfill this task, we decouple it into four subtasks and introduce two implementation solutions called PipVKIE and UniVKIE. PipVKIE sequentially completes the four subtasks in continuous stages, while UniVKIE is improved by unifying all the subtasks into one backbone. Both PipVKIE and UniVKIE leverage multimodal information from vision, text, and coordinates for feature representation. Extensive experiments on one well-defined dataset demonstrate that our solutions can achieve remarkable performance and efficient inference speed.
Anthology ID:
2023.emnlp-industry.51
Volume:
Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Industry Track
Month:
December
Year:
2023
Address:
Singapore
Editors:
Mingxuan Wang, Imed Zitouni
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
532–540
Language:
URL:
https://aclanthology.org/2023.emnlp-industry.51
DOI:
10.18653/v1/2023.emnlp-industry.51
Bibkey:
Cite (ACL):
Siyu An, Ye Liu, Haoyuan Peng, and Di Yin. 2023. VKIE: The Application of Key Information Extraction on Video Text. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 532–540, Singapore. Association for Computational Linguistics.
Cite (Informal):
VKIE: The Application of Key Information Extraction on Video Text (An et al., EMNLP 2023)
Copy Citation:
PDF:
https://preview.aclanthology.org/naacl24-info/2023.emnlp-industry.51.pdf
Video:
 https://preview.aclanthology.org/naacl24-info/2023.emnlp-industry.51.mp4