ARAML: A Stable Adversarial Training Framework for Text Generation
Abstract
Most of the existing generative adversarial networks (GAN) for text generation suffer from the instability of reinforcement learning training algorithms such as policy gradient, leading to unstable performance. To tackle this problem, we propose a novel framework called Adversarial Reward Augmented Maximum Likelihood (ARAML). During adversarial training, the discriminator assigns rewards to samples which are acquired from a stationary distribution near the data rather than the generator’s distribution. The generator is optimized with maximum likelihood estimation augmented by the discriminator’s rewards instead of policy gradient. Experiments show that our model can outperform state-of-the-art text GANs with a more stable training process.- Anthology ID:
- D19-1436
- Volume:
- Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
- Month:
- November
- Year:
- 2019
- Address:
- Hong Kong, China
- Editors:
- Kentaro Inui, Jing Jiang, Vincent Ng, Xiaojun Wan
- Venues:
- EMNLP | IJCNLP
- SIG:
- SIGDAT
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 4271–4281
- Language:
- URL:
- https://aclanthology.org/D19-1436
- DOI:
- 10.18653/v1/D19-1436
- Cite (ACL):
- Pei Ke, Fei Huang, Minlie Huang, and Xiaoyan Zhu. 2019. ARAML: A Stable Adversarial Training Framework for Text Generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4271–4281, Hong Kong, China. Association for Computational Linguistics.
- Cite (Informal):
- ARAML: A Stable Adversarial Training Framework for Text Generation (Ke et al., EMNLP-IJCNLP 2019)
- PDF:
- https://preview.aclanthology.org/teach-a-man-to-fish/D19-1436.pdf
- Code
- kepei1106/ARAML