Abstract
A critical component of competence in language is being able to identify relevant components of an utterance and reply appropriately. In this paper we examine the extent of such dialogue response sensitivity in pre-trained language models, conducting a series of experiments with a particular focus on sensitivity to dynamics involving phenomena of at-issueness and ellipsis. We find that models show clear sensitivity to a distinctive role of embedded clauses, and a general preference for responses that target main clause content of prior utterances. However, the results indicate mixed and generally weak trends with respect to capturing the full range of dynamics involved in targeting at-issue versus not-at-issue content. Additionally, models show fundamental limitations in grasp of the dynamics governing ellipsis, and response selections show clear interference from superficial factors that outweigh the influence of principled discourse constraints.- Anthology ID:
- 2022.coling-1.72
- Volume:
- Proceedings of the 29th International Conference on Computational Linguistics
- Month:
- October
- Year:
- 2022
- Address:
- Gyeongju, Republic of Korea
- Editors:
- Nicoletta Calzolari, Chu-Ren Huang, Hansaem Kim, James Pustejovsky, Leo Wanner, Key-Sun Choi, Pum-Mo Ryu, Hsin-Hsi Chen, Lucia Donatelli, Heng Ji, Sadao Kurohashi, Patrizia Paggio, Nianwen Xue, Seokhwan Kim, Younggyun Hahm, Zhong He, Tony Kyungil Lee, Enrico Santus, Francis Bond, Seung-Hoon Na
- Venue:
- COLING
- SIG:
- Publisher:
- International Committee on Computational Linguistics
- Note:
- Pages:
- 863–874
- Language:
- URL:
- https://aclanthology.org/2022.coling-1.72
- DOI:
- Cite (ACL):
- Sanghee J. Kim, Lang Yu, and Allyson Ettinger. 2022. “No, They Did Not”: Dialogue Response Dynamics in Pre-trained Language Models. In Proceedings of the 29th International Conference on Computational Linguistics, pages 863–874, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
- Cite (Informal):
- “No, They Did Not”: Dialogue Response Dynamics in Pre-trained Language Models (Kim et al., COLING 2022)
- PDF:
- https://preview.aclanthology.org/emnlp22-frontmatter/2022.coling-1.72.pdf