A Hierarchical Encoding-Decoding Scheme for Abstractive Multi-document Summarization
Chenhui Shen, Liying Cheng, Xuan-Phi Nguyen, Yang You, Lidong Bing
Abstract
Pre-trained language models (PLMs) have achieved outstanding achievements in abstractive single-document summarization (SDS). However, such benefits may not fully extend to multi-document summarization (MDS), where the handling of cross-document information is more complex. Previous works either design new MDS architectures or apply PLMs bluntly with concatenated source documents as a reformulated SDS task. While the former does not utilize previous pre-training efforts and may not generalize well across different domains, the latter may not sufficiently attend to the intricate cross-document relationships unique to MDS tasks. Instead, we enforce hierarchy on both the encoder and decoder to better utilize a PLM to facilitate multi-document interactions for the MDS task. Across 10 MDS benchmarks from various domains, our method outperforms or is competitive with the previous best models, including those with additional MDS pre-training or with more parameters. It outperforms its corresponding PLM backbone by up to 3 Rouge-L and is favored by humans.- Anthology ID:
- 2023.findings-emnlp.391
- Volume:
- Findings of the Association for Computational Linguistics: EMNLP 2023
- Month:
- December
- Year:
- 2023
- Address:
- Singapore
- Editors:
- Houda Bouamor, Juan Pino, Kalika Bali
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 5872–5887
- Language:
- URL:
- https://aclanthology.org/2023.findings-emnlp.391
- DOI:
- 10.18653/v1/2023.findings-emnlp.391
- Cite (ACL):
- Chenhui Shen, Liying Cheng, Xuan-Phi Nguyen, Yang You, and Lidong Bing. 2023. A Hierarchical Encoding-Decoding Scheme for Abstractive Multi-document Summarization. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 5872–5887, Singapore. Association for Computational Linguistics.
- Cite (Informal):
- A Hierarchical Encoding-Decoding Scheme for Abstractive Multi-document Summarization (Shen et al., Findings 2023)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-1/2023.findings-emnlp.391.pdf