Abstract
Understanding perspectival language is important for applications like dialogue systems and human-robot interaction. We propose a probe task that explores how well language models understand spatial perspective. We present a dataset for evaluating perspective inference in English, ProSPer, and use it to explore how humans and Transformer-based language models infer perspective. Although the best bidirectional model performs similarly to humans, they display different strengths: humans outperform neural networks in conversational contexts, while RoBERTa excels at written genres.- Anthology ID:
- 2021.blackboxnlp-1.8
- Volume:
- Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP
- Month:
- November
- Year:
- 2021
- Address:
- Punta Cana, Dominican Republic
- Venue:
- BlackboxNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 95–135
- Language:
- URL:
- https://aclanthology.org/2021.blackboxnlp-1.8
- DOI:
- 10.18653/v1/2021.blackboxnlp-1.8
- Cite (ACL):
- Tessa Masis and Carolyn Anderson. 2021. ProSPer: Probing Human and Neural Network Language Model Understanding of Spatial Perspective. In Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, pages 95–135, Punta Cana, Dominican Republic. Association for Computational Linguistics.
- Cite (Informal):
- ProSPer: Probing Human and Neural Network Language Model Understanding of Spatial Perspective (Masis & Anderson, BlackboxNLP 2021)
- PDF:
- https://preview.aclanthology.org/ingestion-script-update/2021.blackboxnlp-1.8.pdf
- Code
- canders1/prosper