Abstract
We investigate the stability of narrative schemas (Chambers and Jurafsky, 2009) automatically induced from a news corpus, representing recurring narratives in a corpus. If such techniques produce meaningful results, we should expect that small changes to the corpus will result in only small changes to the induced schemas. We describe experiments involving successive ablation of a corpus and cross-validation at each stage of ablation, on schemas generated by three different techniques over a general news corpus and topically-specific subcorpora. We also develop a method for evaluating the similarity between sets of narrative schemas, and thus the stability of the schema induction algorithms. This stability analysis affirms the heterogeneous/homogeneous document category hypothesis first presented in Simonson and Davis (2016), whose technique is problematically limited. Additionally, increased ablation leads to increasing stability, so the smaller the remaining corpus, the more stable schema generation appears to be. We surmise that as a corpus grows larger, novel and more varied narratives continue to appear and stability declines, though at some point this decline levels off as new additions to the corpus consist essentially of “more of the same.”- Anthology ID:
- C18-1311
- Volume:
- Proceedings of the 27th International Conference on Computational Linguistics
- Month:
- August
- Year:
- 2018
- Address:
- Santa Fe, New Mexico, USA
- Editors:
- Emily M. Bender, Leon Derczynski, Pierre Isabelle
- Venue:
- COLING
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 3670–3680
- Language:
- URL:
- https://aclanthology.org/C18-1311
- DOI:
- Cite (ACL):
- Dan Simonson and Anthony Davis. 2018. Narrative Schema Stability in News Text. In Proceedings of the 27th International Conference on Computational Linguistics, pages 3670–3680, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
- Cite (Informal):
- Narrative Schema Stability in News Text (Simonson & Davis, COLING 2018)
- PDF:
- https://preview.aclanthology.org/proper-vol2-ingestion/C18-1311.pdf
- Data
- New York Times Annotated Corpus