Composing a Picture Book by Automatic Story Understanding and Visualization

Xiaoyu Qi, Ruihua Song, Chunting Wang, Jin Zhou, Tetsuya Sakai


Abstract
Pictures can enrich storytelling experiences. We propose a framework that can automatically compose a picture book by understanding story text and visualizing it with painting elements, i.e., characters and backgrounds. For story understanding, we extract key information from a story on both sentence level and paragraph level, including characters, scenes and actions. These concepts are organized and visualized in a way that depicts the development of a story. We collect a set of Chinese stories for children and apply our approach to compose pictures for stories. Extensive experiments are conducted towards story event extraction for visualization to demonstrate the effectiveness of our method.
Anthology ID:
W19-3401
Volume:
Proceedings of the Second Workshop on Storytelling
Month:
August
Year:
2019
Address:
Florence, Italy
Editors:
Francis Ferraro, Ting-Hao ‘Kenneth’ Huang, Stephanie M. Lukin, Margaret Mitchell
Venue:
Story-NLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1–10
Language:
URL:
https://aclanthology.org/W19-3401
DOI:
10.18653/v1/W19-3401
Bibkey:
Cite (ACL):
Xiaoyu Qi, Ruihua Song, Chunting Wang, Jin Zhou, and Tetsuya Sakai. 2019. Composing a Picture Book by Automatic Story Understanding and Visualization. In Proceedings of the Second Workshop on Storytelling, pages 1–10, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
Composing a Picture Book by Automatic Story Understanding and Visualization (Qi et al., Story-NLP 2019)
Copy Citation:
PDF:
https://preview.aclanthology.org/add_acl24_videos/W19-3401.pdf