Disentangling Indirect Answers to Yes-No Questions in Real Conversations
Krishna Sanagavarapu, Jathin Singaraju, Anusha Kakileti, Anirudh Kaza, Aaron Mathews, Helen Li, Nathan Brito, Eduardo Blanco
Abstract
In this paper, we explore the task of determining indirect answers to yes-no questions in real conversations. We work with transcripts of phone conversations in the Switchboard Dialog Act (SwDA) corpus and create SwDA-IndirectAnswers (SwDA-IA), a subset of SwDA consisting of all conversations containing a yes-no question with an indirect answer. We annotate the underlying direct answers to the yes-no questions (yes, probably yes, middle, probably no, or no). We show that doing so requires taking into account conversation context: the indirect answer alone is insufficient to determine the ground truth. Experimental results also show that taking into account context is beneficial. More importantly, our results demonstrate that existing corpora with synthetic indirect answers to yes-no questions are not beneficial when working with real conversations. Our best models outperform the majority baseline by a substantial margin, but the task remains a challenge (F1: 0.46).- Anthology ID:
- 2022.naacl-main.345
- Volume:
- Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
- Month:
- July
- Year:
- 2022
- Address:
- Seattle, United States
- Editors:
- Marine Carpuat, Marie-Catherine de Marneffe, Ivan Vladimir Meza Ruiz
- Venue:
- NAACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 4677–4695
- Language:
- URL:
- https://aclanthology.org/2022.naacl-main.345
- DOI:
- 10.18653/v1/2022.naacl-main.345
- Cite (ACL):
- Krishna Sanagavarapu, Jathin Singaraju, Anusha Kakileti, Anirudh Kaza, Aaron Mathews, Helen Li, Nathan Brito, and Eduardo Blanco. 2022. Disentangling Indirect Answers to Yes-No Questions in Real Conversations. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4677–4695, Seattle, United States. Association for Computational Linguistics.
- Cite (Informal):
- Disentangling Indirect Answers to Yes-No Questions in Real Conversations (Sanagavarapu et al., NAACL 2022)
- PDF:
- https://preview.aclanthology.org/dois-2013-emnlp/2022.naacl-main.345.pdf
- Code
- krishna-chaitanya-sanagavarapu/swda-ia
- Data
- BoolQ, MultiNLI