Measuring and Mitigating Media Outlet Name Bias in Large Language Models

Seong-Jin Park, Kang-Min Kim


Abstract
Large language models (LLMs) have achieved remarkable performance across diverse natural language processing tasks, but concerns persist regarding their potential political biases. While prior research has extensively explored political biases in LLMs’ text generation and perception, limited attention has been devoted to biases associated with media outlet names. In this study, we systematically investigate the presence of media outlet name biases in LLMs and evaluate their impact on downstream tasks, such as political bias prediction and news summarization. Our findings demonstrate that LLMs consistently exhibit biases toward the known political leanings of media outlets, with variations across model families and scales. We propose a novel metric to quantify media outlet name biases in LLMs and leverage this metric to develop an automated prompt optimization framework. Our framework effectively mitigates media outlet name biases, offering a scalable approach to enhancing the fairness of LLMs in news-related applications.
Anthology ID:
2025.emnlp-main.1513
Volume:
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Month:
November
Year:
2025
Address:
Suzhou, China
Editors:
Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, Violet Peng
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
29766–29785
Language:
URL:
https://preview.aclanthology.org/author-page-yu-wang-polytechnic/2025.emnlp-main.1513/
DOI:
10.18653/v1/2025.emnlp-main.1513
Bibkey:
Cite (ACL):
Seong-Jin Park and Kang-Min Kim. 2025. Measuring and Mitigating Media Outlet Name Bias in Large Language Models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 29766–29785, Suzhou, China. Association for Computational Linguistics.
Cite (Informal):
Measuring and Mitigating Media Outlet Name Bias in Large Language Models (Park & Kim, EMNLP 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/author-page-yu-wang-polytechnic/2025.emnlp-main.1513.pdf
Checklist:
 2025.emnlp-main.1513.checklist.pdf