Colors in Context: A Pragmatic Neural Model for Grounded Language Understanding
Will Monroe, Robert X.D. Hawkins, Noah D. Goodman, Christopher Potts
Abstract
We present a model of pragmatic referring expression interpretation in a grounded communication task (identifying colors from descriptions) that draws upon predictions from two recurrent neural network classifiers, a speaker and a listener, unified by a recursive pragmatic reasoning framework. Experiments show that this combined pragmatic model interprets color descriptions more accurately than the classifiers from which it is built, and that much of this improvement results from combining the speaker and listener perspectives. We observe that pragmatic reasoning helps primarily in the hardest cases: when the model must distinguish very similar colors, or when few utterances adequately express the target color. Our findings make use of a newly-collected corpus of human utterances in color reference games, which exhibit a variety of pragmatic behaviors. We also show that the embedded speaker model reproduces many of these pragmatic behaviors.- Anthology ID:
- Q17-1023
- Volume:
- Transactions of the Association for Computational Linguistics, Volume 5
- Month:
- Year:
- 2017
- Address:
- Cambridge, MA
- Editors:
- Lillian Lee, Mark Johnson, Kristina Toutanova
- Venue:
- TACL
- SIG:
- Publisher:
- MIT Press
- Note:
- Pages:
- 325–338
- Language:
- URL:
- https://aclanthology.org/Q17-1023
- DOI:
- 10.1162/tacl_a_00064
- Cite (ACL):
- Will Monroe, Robert X.D. Hawkins, Noah D. Goodman, and Christopher Potts. 2017. Colors in Context: A Pragmatic Neural Model for Grounded Language Understanding. Transactions of the Association for Computational Linguistics, 5:325–338.
- Cite (Informal):
- Colors in Context: A Pragmatic Neural Model for Grounded Language Understanding (Monroe et al., TACL 2017)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-1/Q17-1023.pdf