Abstract
Natural language has recently been explored as a new medium of supervision for training machine learning models. Here, we explore learning classification tasks using language in a conversational setting – where the automated learner does not simply receive language input from a teacher, but can proactively engage the teacher by asking questions. We present a reinforcement learning framework, where the learner’s actions correspond to question types and the reward for asking a question is based on how the teacher’s response changes performance of the resulting machine learning model on the learning task. In this framework, learning good question-asking strategies corresponds to asking sequences of questions that maximize the cumulative (discounted) reward, and hence quickly lead to effective classifiers. Empirical analysis across three domains shows that learned question-asking strategies expedite classifier training by asking appropriate questions at different points in the learning process. The approach allows learning classifiers from a blend of strategies, including learning from observations, explanations and clarifications.- Anthology ID:
- D19-1426
- Volume:
- Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
- Month:
- November
- Year:
- 2019
- Address:
- Hong Kong, China
- Editors:
- Kentaro Inui, Jing Jiang, Vincent Ng, Xiaojun Wan
- Venues:
- EMNLP | IJCNLP
- SIG:
- SIGDAT
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 4164–4174
- Language:
- URL:
- https://aclanthology.org/D19-1426
- DOI:
- 10.18653/v1/D19-1426
- Cite (ACL):
- Shashank Srivastava, Igor Labutov, and Tom Mitchell. 2019. Learning to Ask for Conversational Machine Learning. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4164–4174, Hong Kong, China. Association for Computational Linguistics.
- Cite (Informal):
- Learning to Ask for Conversational Machine Learning (Srivastava et al., EMNLP-IJCNLP 2019)
- PDF:
- https://preview.aclanthology.org/teach-a-man-to-fish/D19-1426.pdf