Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Information

Sudha Rao, Hal Daumé III

[How to correct problems with metadata yourself]


Abstract
Inquiry is fundamental to communication, and machines cannot effectively collaborate with humans unless they can ask questions. In this work, we build a neural network model for the task of ranking clarification questions. Our model is inspired by the idea of expected value of perfect information: a good question is one whose expected answer will be useful. We study this problem using data from StackExchange, a plentiful online resource in which people routinely ask clarifying questions to posts so that they can better offer assistance to the original poster. We create a dataset of clarification questions consisting of 77K posts paired with a clarification question (and answer) from three domains of StackExchange: askubuntu, unix and superuser. We evaluate our model on 500 samples of this dataset against expert human judgments and demonstrate significant improvements over controlled baselines.
Anthology ID:
P18-1255
Volume:
Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
July
Year:
2018
Address:
Melbourne, Australia
Editors:
Iryna Gurevych, Yusuke Miyao
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2737–2746
Language:
URL:
https://aclanthology.org/P18-1255
DOI:
10.18653/v1/P18-1255
Bibkey:
Cite (ACL):
Sudha Rao and Hal Daumé III. 2018. Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Information. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2737–2746, Melbourne, Australia. Association for Computational Linguistics.
Cite (Informal):
Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Information (Rao & Daumé III, ACL 2018)
Copy Citation:
PDF:
https://preview.aclanthology.org/teach-a-man-to-fish/P18-1255.pdf
Note:
 P18-1255.Notes.pdf
Presentation:
 P18-1255.Presentation.pdf
Video:
 https://preview.aclanthology.org/teach-a-man-to-fish/P18-1255.mp4
Code
 raosudha89/ranking_clarification_questions