Speculative Sampling in Variational Autoencoders for Dialogue Response Generation
Shoetsu Sato, Naoki Yoshinaga, Masashi Toyoda, Masaru Kitsuregawa
Abstract
Variational autoencoders have been studied as a promising approach to model one-to-many mappings from context to response in chat response generation. However, they often fail to learn proper mappings. One of the reasons for this failure is the discrepancy between a response and a latent variable sampled from an approximated distribution in training. Inappropriately sampled latent variables hinder models from constructing a modulated latent space. As a result, the models stop handling uncertainty in conversations. To resolve that, we propose speculative sampling of latent variables. Our method chooses the most probable one from redundantly sampled latent variables for tying up the variable with a given response. We confirm the efficacy of our method in response generation with massive dialogue data constructed from Twitter posts.- Anthology ID:
- 2021.findings-emnlp.407
- Volume:
- Findings of the Association for Computational Linguistics: EMNLP 2021
- Month:
- November
- Year:
- 2021
- Address:
- Punta Cana, Dominican Republic
- Venue:
- Findings
- SIG:
- SIGDAT
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 4739–4745
- Language:
- URL:
- https://aclanthology.org/2021.findings-emnlp.407
- DOI:
- 10.18653/v1/2021.findings-emnlp.407
- Cite (ACL):
- Shoetsu Sato, Naoki Yoshinaga, Masashi Toyoda, and Masaru Kitsuregawa. 2021. Speculative Sampling in Variational Autoencoders for Dialogue Response Generation. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 4739–4745, Punta Cana, Dominican Republic. Association for Computational Linguistics.
- Cite (Informal):
- Speculative Sampling in Variational Autoencoders for Dialogue Response Generation (Sato et al., Findings 2021)
- PDF:
- https://preview.aclanthology.org/ingestion-script-update/2021.findings-emnlp.407.pdf
- Code
- jack-and-rozz/speculative_sampling