An Empirical Analysis of Diversity in Argument Summarization
Michiel Van Der Meer, Piek Vossen, Catholijn Jonker, Pradeep Murukannaiah
Abstract
Presenting high-level arguments is a crucial task for fostering participation in online societal discussions. Current argument summarization approaches miss an important facet of this task—capturing diversity—which is important for accommodating multiple perspectives. We introduce three aspects of diversity: those of opinions, annotators, and sources. We evaluate approaches to a popular argument summarization task called Key Point Analysis, which shows how these approaches struggle to (1) represent arguments shared by few people, (2) deal with data from various sources, and (3) align with subjectivity in human-provided annotations. We find that both general-purpose LLMs and dedicated KPA models exhibit this behavior, but have complementary strengths. Further, we observe that diversification of training data may ameliorate generalization in zero-shot cases. Addressing diversity in argument summarization requires a mix of strategies to deal with subjectivity.- Anthology ID:
- 2024.eacl-long.123
- Volume:
- Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
- Month:
- March
- Year:
- 2024
- Address:
- St. Julian’s, Malta
- Editors:
- Yvette Graham, Matthew Purver
- Venue:
- EACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 2028–2045
- Language:
- URL:
- https://aclanthology.org/2024.eacl-long.123
- DOI:
- Cite (ACL):
- Michiel Van Der Meer, Piek Vossen, Catholijn Jonker, and Pradeep Murukannaiah. 2024. An Empirical Analysis of Diversity in Argument Summarization. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2028–2045, St. Julian’s, Malta. Association for Computational Linguistics.
- Cite (Informal):
- An Empirical Analysis of Diversity in Argument Summarization (Van Der Meer et al., EACL 2024)
- PDF:
- https://preview.aclanthology.org/bionlp-24-ingestion/2024.eacl-long.123.pdf