Improving Neural Story Generation by Targeted Common Sense Grounding
Huanru Henry Mao, Bodhisattwa Prasad Majumder, Julian McAuley, Garrison Cottrell
Abstract
Stories generated with neural language models have shown promise in grammatical and stylistic consistency. However, the generated stories are still lacking in common sense reasoning, e.g., they often contain sentences deprived of world knowledge. We propose a simple multi-task learning scheme to achieve quantitatively better common sense reasoning in language models by leveraging auxiliary training signals from datasets designed to provide common sense grounding. When combined with our two-stage fine-tuning pipeline, our method achieves improved common sense reasoning and state-of-the-art perplexity on the WritingPrompts (Fan et al., 2018) story generation dataset.- Anthology ID:
- D19-1615
- Volume:
- Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
- Month:
- November
- Year:
- 2019
- Address:
- Hong Kong, China
- Editors:
- Kentaro Inui, Jing Jiang, Vincent Ng, Xiaojun Wan
- Venues:
- EMNLP | IJCNLP
- SIG:
- SIGDAT
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 5988–5993
- Language:
- URL:
- https://aclanthology.org/D19-1615
- DOI:
- 10.18653/v1/D19-1615
- Cite (ACL):
- Huanru Henry Mao, Bodhisattwa Prasad Majumder, Julian McAuley, and Garrison Cottrell. 2019. Improving Neural Story Generation by Targeted Common Sense Grounding. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5988–5993, Hong Kong, China. Association for Computational Linguistics.
- Cite (Informal):
- Improving Neural Story Generation by Targeted Common Sense Grounding (Mao et al., EMNLP-IJCNLP 2019)
- PDF:
- https://preview.aclanthology.org/naacl24-info/D19-1615.pdf
- Code
- calclavia/story-generation
- Data
- BookCorpus, SWAG, StoryCloze, WebText, WritingPrompts