REFER: Mitigating Bias in Opinion Summarisation via Frequency Framed Prompting

Nannan Huang, Haytham M. Fayek, Xiuzhen Zhang


Abstract
Individuals express diverse opinions, a fair summary should represent these viewpoints comprehensively.Previous research on fairness in opinion summarisation using large language models (LLMs) relied on hyperparameter tuning or providing ground truth distributional information in prompts. However, these methods face practical limitations: end-users rarely modify default model parameters, and accurate distributional information is often unavailable. Building upon cognitive science research demonstrating that frequency-based representations reduce systematic biases in human statistical reasoning by making reference classes explicit and reducing cognitive load, this study investigates whether frequency framed prompting (REFER) can similarly enhance fairness in LLM opinion summarisation. Through systematic experimentation with different prompting frameworks, we adapted techniques known to improve human reasoning to elicit more effective information processing in language models compared to abstract probabilistic representations. Our results demonstrate that REFER enhances fairness in language models when summarising opinions. This effect is particularly pronounced in larger language models and using stronger reasoning instructions.
Anthology ID:
2025.newsum-main.6
Volume:
Proceedings of The 5th New Frontiers in Summarization Workshop
Month:
November
Year:
2025
Address:
Hybrid
Editors:
Yue Dong, Wen Xiao, Haopeng Zhang, Rui Zhang, Ori Ernst, Lu Wang, Fei Liu
Venues:
NewSum | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
74–93
Language:
URL:
https://preview.aclanthology.org/ingest-emnlp/2025.newsum-main.6/
DOI:
Bibkey:
Cite (ACL):
Nannan Huang, Haytham M. Fayek, and Xiuzhen Zhang. 2025. REFER: Mitigating Bias in Opinion Summarisation via Frequency Framed Prompting. In Proceedings of The 5th New Frontiers in Summarization Workshop, pages 74–93, Hybrid. Association for Computational Linguistics.
Cite (Informal):
REFER: Mitigating Bias in Opinion Summarisation via Frequency Framed Prompting (Huang et al., NewSum 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-emnlp/2025.newsum-main.6.pdf