Length Control in Abstractive Summarization by Pretraining Information Selection

Yizhu Liu, Qi Jia, Kenny Zhu


Abstract
Previous length-controllable summarization models mostly control lengths at the decoding stage, whereas the encoding or the selection of information from the source document is not sensitive to the designed length. They also tend to generate summaries as long as those in the training data. In this paper, we propose a length-aware attention mechanism (LAAM) to adapt the encoding of the source based on the desired length. Our approach works by training LAAM on a summary length balanced dataset built from the original training data, and then fine-tuning as usual. Results show that this approach is effective in generating high-quality summaries with desired lengths and even those short lengths never seen in the original training set.
Anthology ID:
2022.acl-long.474
Volume:
Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
May
Year:
2022
Address:
Dublin, Ireland
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
6885–6895
Language:
URL:
https://aclanthology.org/2022.acl-long.474
DOI:
10.18653/v1/2022.acl-long.474
Bibkey:
Cite (ACL):
Yizhu Liu, Qi Jia, and Kenny Zhu. 2022. Length Control in Abstractive Summarization by Pretraining Information Selection. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6885–6895, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
Length Control in Abstractive Summarization by Pretraining Information Selection (Liu et al., ACL 2022)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-script-update/2022.acl-long.474.pdf
Software:
 2022.acl-long.474.software.zip
Code
 yizhuliu/lengthcontrol