Abstract
Recently, we can obtain a practical abstractive document summarization model by fine-tuning a pre-trained language model (PLM). Since the pre-training for PLMs does not consider summarization-specific information such as the target summary length, there is a gap between the pre-training and fine-tuning for PLMs in summarization tasks. To fill the gap, we propose a method for enabling the model to understand the summarization-specific information by predicting the summary length in the encoder and generating a summary of the predicted length in the decoder in fine-tuning. Experimental results on the WikiHow, NYT, and CNN/DM datasets showed that our methods improve ROUGE scores from BART by generating summaries of appropriate lengths. Further, we observed about 3.0, 1,5, and 3.1 point improvements for ROUGE-1, -2, and -L, respectively, from GSum on the WikiHow dataset. Human evaluation results also showed that our methods improve the informativeness and conciseness of summaries.- Anthology ID:
- 2023.findings-eacl.45
- Volume:
- Findings of the Association for Computational Linguistics: EACL 2023
- Month:
- May
- Year:
- 2023
- Address:
- Dubrovnik, Croatia
- Editors:
- Andreas Vlachos, Isabelle Augenstein
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 618–624
- Language:
- URL:
- https://aclanthology.org/2023.findings-eacl.45
- DOI:
- 10.18653/v1/2023.findings-eacl.45
- Cite (ACL):
- Jingun Kwon, Hidetaka Kamigaito, and Manabu Okumura. 2023. Abstractive Document Summarization with Summary-length Prediction. In Findings of the Association for Computational Linguistics: EACL 2023, pages 618–624, Dubrovnik, Croatia. Association for Computational Linguistics.
- Cite (Informal):
- Abstractive Document Summarization with Summary-length Prediction (Kwon et al., Findings 2023)
- PDF:
- https://preview.aclanthology.org/proper-vol2-ingestion/2023.findings-eacl.45.pdf