Abstract
Opinion summarisation is a task that aims to condense the information presented in the source documents while retaining the core message and opinions. A summary that only represents the majority opinions will leave the minority opinions unrepresented in the summary. In this paper, we use the stance towards a certain target as an opinion. We study bias in opinion summarisation from the perspective of opinion diversity, which measures whether the model generated summary can cover a diverse set of opinions. In addition, we examine opinion similarity, a measure of how closely related two opinions are in terms of their stance on a given topic, and its relationship with opinion diversity. Through the lense of stances towards a topic, we examine opinion diversity and similarity using three debatable topics under COVID-19. Experimental results on these topics revealed that a higher degree of similarity of opinions did not indicate good diversity or fairly cover the various opinions originally presented in the source documents. We found that BART and ChatGPT can better capture diverse opinions presented in the source documents.- Anthology ID:
- 2023.wassa-1.14
- Volume:
- Proceedings of the 13th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis
- Month:
- July
- Year:
- 2023
- Address:
- Toronto, Canada
- Editors:
- Jeremy Barnes, Orphée De Clercq, Roman Klinger
- Venue:
- WASSA
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 149–161
- Language:
- URL:
- https://aclanthology.org/2023.wassa-1.14
- DOI:
- 10.18653/v1/2023.wassa-1.14
- Cite (ACL):
- Nannan Huang, Lin Tian, Haytham Fayek, and Xiuzhen Zhang. 2023. Examining Bias in Opinion Summarisation through the Perspective of Opinion Diversity. In Proceedings of the 13th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis, pages 149–161, Toronto, Canada. Association for Computational Linguistics.
- Cite (Informal):
- Examining Bias in Opinion Summarisation through the Perspective of Opinion Diversity (Huang et al., WASSA 2023)
- PDF:
- https://preview.aclanthology.org/emnlp-22-attachments/2023.wassa-1.14.pdf