A Prospective-Performance Network to Alleviate Myopia in Beam Search for Response Generation
Zongsheng Wang, Yunzhi Bai, Bowen Wu, Zhen Xu, Zhuoran Wang, Baoxun Wang
Abstract
Generative dialog models usually adopt beam search as the inference method to generate responses. However, small-width beam search only focuses on the limited current optima. This deficiency named as myopic bias ultimately suppresses the diversity and probability of generated responses. Although increasing the beam width mitigates the myopic bias, it also proportionally slows down the inference efficiency. To alleviate the myopic bias in small-width beam search, this paper proposes a Prospective-Performance Network (PPN) to predict the future reward of the given partially-generated response, and the future reward is defined by the expectation of the partial response appearing in the top-ranked responses given by a larger-width beam search. Enhanced by PPN, the decoder can promote the results with great potential during the beam search phase. The experimental results on both Chinese and English corpora show that our method is promising to increase the quality and diversity of generated responses, with inference efficiency well maintained.- Anthology ID:
- C18-1306
- Volume:
- Proceedings of the 27th International Conference on Computational Linguistics
- Month:
- August
- Year:
- 2018
- Address:
- Santa Fe, New Mexico, USA
- Editors:
- Emily M. Bender, Leon Derczynski, Pierre Isabelle
- Venue:
- COLING
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 3608–3618
- Language:
- URL:
- https://aclanthology.org/C18-1306
- DOI:
- Cite (ACL):
- Zongsheng Wang, Yunzhi Bai, Bowen Wu, Zhen Xu, Zhuoran Wang, and Baoxun Wang. 2018. A Prospective-Performance Network to Alleviate Myopia in Beam Search for Response Generation. In Proceedings of the 27th International Conference on Computational Linguistics, pages 3608–3618, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
- Cite (Informal):
- A Prospective-Performance Network to Alleviate Myopia in Beam Search for Response Generation (Wang et al., COLING 2018)
- PDF:
- https://preview.aclanthology.org/proper-vol2-ingestion/C18-1306.pdf
- Data
- OpenSubtitles