Abstract
Pointer Generators have been the de facto standard for modern summarization systems. However, this architecture faces two major drawbacks: Firstly, the pointer is limited to copying the exact words while ignoring possible inflections or abstractions, which restricts its power of capturing richer latent alignment. Secondly, the copy mechanism results in a strong bias towards extractive generations, where most sentences are produced by simply copying from the source text. In this paper, we address these problems by allowing the model to “edit” pointed tokens instead of always hard copying them. The editing is performed by transforming the pointed word vector into a target space with a learned relation embedding. On three large-scale summarization dataset, we show the model is able to (1) capture more latent alignment relations than exact word matches, (2) improve word alignment accuracy, allowing for better model interpretation and controlling, (3) generate higher-quality summaries validated by both qualitative and quantitative evaluations and (4) bring more abstraction to the generated summaries.- Anthology ID:
- D19-1390
- Volume:
- Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
- Month:
- November
- Year:
- 2019
- Address:
- Hong Kong, China
- Editors:
- Kentaro Inui, Jing Jiang, Vincent Ng, Xiaojun Wan
- Venues:
- EMNLP | IJCNLP
- SIG:
- SIGDAT
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 3762–3773
- Language:
- URL:
- https://aclanthology.org/D19-1390
- DOI:
- 10.18653/v1/D19-1390
- Cite (ACL):
- Xiaoyu Shen, Yang Zhao, Hui Su, and Dietrich Klakow. 2019. Improving Latent Alignment in Text Summarization by Generalizing the Pointer Generator. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3762–3773, Hong Kong, China. Association for Computational Linguistics.
- Cite (Informal):
- Improving Latent Alignment in Text Summarization by Generalizing the Pointer Generator (Shen et al., EMNLP-IJCNLP 2019)
- PDF:
- https://preview.aclanthology.org/improve-issue-templates/D19-1390.pdf