Abstract
We propose a general class of language models that treat reference as discrete stochastic latent variables. This decision allows for the creation of entity mentions by accessing external databases of referents (required by, e.g., dialogue generation) or past internal state (required to explicitly model coreferentiality). Beyond simple copying, our coreference model can additionally refer to a referent using varied mention forms (e.g., a reference to “Jane” can be realized as “she”), a characteristic feature of reference in natural languages. Experiments on three representative applications show our model variants outperform models based on deterministic attention and standard language modeling baselines.- Anthology ID:
- D17-1197
- Volume:
- Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing
- Month:
- September
- Year:
- 2017
- Address:
- Copenhagen, Denmark
- Editors:
- Martha Palmer, Rebecca Hwa, Sebastian Riedel
- Venue:
- EMNLP
- SIG:
- SIGDAT
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1850–1859
- Language:
- URL:
- https://aclanthology.org/D17-1197
- DOI:
- 10.18653/v1/D17-1197
- Cite (ACL):
- Zichao Yang, Phil Blunsom, Chris Dyer, and Wang Ling. 2017. Reference-Aware Language Models. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1850–1859, Copenhagen, Denmark. Association for Computational Linguistics.
- Cite (Informal):
- Reference-Aware Language Models (Yang et al., EMNLP 2017)
- PDF:
- https://preview.aclanthology.org/proper-vol2-ingestion/D17-1197.pdf