Topic-Controllable Summarization: Topic-Aware Evaluation and Transformer Methods
Abstract
Topic-controllable summarization is an emerging research area with a wide range of potential applications. However, existing approaches suffer from significant limitations. For example, the majority of existing methods built upon recurrent architectures, which can significantly limit their performance compared to more recent Transformer-based architectures, while they also require modifications to the model’s architecture for controlling the topic. At the same time, there is currently no established evaluation metric designed specifically for topic-controllable summarization. This work proposes a new topic-oriented evaluation measure to automatically evaluate the generated summaries based on the topic affinity between the generated summary and the desired topic. The reliability of the proposed measure is demonstrated through appropriately designed human evaluation. In addition, we adapt topic embeddings to work with powerful Transformer architectures and propose a novel and efficient approach for guiding the summary generation through control tokens. Experimental results reveal that control tokens can achieve better performance compared to more complicated embedding-based approaches while also being significantly faster.- Anthology ID:
- 2024.lrec-main.1415
- Volume:
- Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)
- Month:
- May
- Year:
- 2024
- Address:
- Torino, Italia
- Editors:
- Nicoletta Calzolari, Min-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, Nianwen Xue
- Venues:
- LREC | COLING
- SIG:
- Publisher:
- ELRA and ICCL
- Note:
- Pages:
- 16282–16292
- Language:
- URL:
- https://aclanthology.org/2024.lrec-main.1415
- DOI:
- Cite (ACL):
- Tatiana Passali and Grigorios Tsoumakas. 2024. Topic-Controllable Summarization: Topic-Aware Evaluation and Transformer Methods. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 16282–16292, Torino, Italia. ELRA and ICCL.
- Cite (Informal):
- Topic-Controllable Summarization: Topic-Aware Evaluation and Transformer Methods (Passali & Tsoumakas, LREC-COLING 2024)
- PDF:
- https://preview.aclanthology.org/teach-a-man-to-fish/2024.lrec-main.1415.pdf