Generating Animations from Screenplays
Yeyao Zhang, Eleftheria Tsipidi, Sasha Schriber, Mubbasir Kapadia, Markus Gross, Ashutosh Modi
Abstract
Automatically generating animation from natural language text finds application in a number of areas e.g. movie script writing, instructional videos, and public safety. However, translating natural language text into animation is a challenging task. Existing text-to-animation systems can handle only very simple sentences, which limits their applications. In this paper, we develop a text-to-animation system which is capable of handling complex sentences. We achieve this by introducing a text simplification step into the process. Building on an existing animation generation system for screenwriting, we create a robust NLP pipeline to extract information from screenplays and map them to the system’s knowledge base. We develop a set of linguistic transformation rules that simplify complex sentences. Information extracted from the simplified sentences is used to generate a rough storyboard and video depicting the text. Our sentence simplification module outperforms existing systems in terms of BLEU and SARI metrics. We further evaluated our system via a user study: 68% participants believe that our system generates reasonable animation from input screenplays.- Anthology ID:
- S19-1032
- Volume:
- Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM 2019)
- Month:
- June
- Year:
- 2019
- Address:
- Minneapolis, Minnesota
- Editors:
- Rada Mihalcea, Ekaterina Shutova, Lun-Wei Ku, Kilian Evang, Soujanya Poria
- Venue:
- *SEM
- SIGs:
- SIGLEX | SIGSEM
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 292–307
- Language:
- URL:
- https://aclanthology.org/S19-1032
- DOI:
- 10.18653/v1/S19-1032
- Cite (ACL):
- Yeyao Zhang, Eleftheria Tsipidi, Sasha Schriber, Mubbasir Kapadia, Markus Gross, and Ashutosh Modi. 2019. Generating Animations from Screenplays. In Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM 2019), pages 292–307, Minneapolis, Minnesota. Association for Computational Linguistics.
- Cite (Informal):
- Generating Animations from Screenplays (Zhang et al., *SEM 2019)
- PDF:
- https://preview.aclanthology.org/ingest-acl-2023-videos/S19-1032.pdf