Abstract
Aspect-based summarization has seen significant advancements, especially in structured text. Yet, summarizing disordered, large-scale texts, like those found in social media and customer feedback, remains a significant challenge. Current research largely targets predefined aspects within structured texts, neglecting the complexities of dynamic and disordered environments. Addressing this gap, we introduce Disordered-DABS, a novel benchmark for dynamic aspect-based summarization tailored to unstructured text. Developed by adapting existing datasets for cost-efficiency and scalability, our comprehensive experiments and detailed human evaluations reveal that Disordered-DABS poses unique challenges to contemporary summarization models, including state-of-the-art language models such as GPT-3.5.- Anthology ID:
- 2024.findings-emnlp.24
- Volume:
- Findings of the Association for Computational Linguistics: EMNLP 2024
- Month:
- November
- Year:
- 2024
- Address:
- Miami, Florida, USA
- Editors:
- Yaser Al-Onaizan, Mohit Bansal, Yun-Nung Chen
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 416–431
- Language:
- URL:
- https://aclanthology.org/2024.findings-emnlp.24
- DOI:
- 10.18653/v1/2024.findings-emnlp.24
- Cite (ACL):
- Xiaobo Guo and Soroush Vosoughi. 2024. Disordered-DABS: A Benchmark for Dynamic Aspect-Based Summarization in Disordered Texts. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 416–431, Miami, Florida, USA. Association for Computational Linguistics.
- Cite (Informal):
- Disordered-DABS: A Benchmark for Dynamic Aspect-Based Summarization in Disordered Texts (Guo & Vosoughi, Findings 2024)
- PDF:
- https://preview.aclanthology.org/dois-2013-emnlp/2024.findings-emnlp.24.pdf