Fair or Framed? Political Bias in News Articles Generated by LLMs

Junho Yoo, Youhyun Shin


Abstract
Despite biases in Large Language Models (LLMs) being widely researched, systematic explorations of political biases in news article generation tasks remain underexplored. This study evaluates political bias across seven LLMs by leveraging our PublicViews dataset-extracted from the TwinViews-13K corpus-comprising 31 topics and 31,692 statements. We analyze 10,850 articles, finding left-leaning bias persists in generation tasks, with neutral content remaining rare even under balanced opinion settings. Models exhibit asymmetric behavior in minority opinion scenarios, amplifying preferred viewpoints when in minority while conforming to majority opinions otherwise. Notably, all models employ ‘stance-flipping quotations” (altering supporters’ statements to express opposite viewpoints) in 33-38% of quotations despite explicit instructions against distortion. Consistent with prior research, increased model size failed to enhance neutrality. This research measures political bias in LLM-generated news, analyzes its mechanisms, and reveals how opinion distribution and explicitness affect bias expression. Our results highlight how LLMs can introduce unintended political bias in generative contexts. We publicly release our PublicViews corpus and code at https://anonymous.4open.science/r/Fair-or-Framed-46F1.
Anthology ID:
2025.emnlp-main.856
Volume:
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Month:
November
Year:
2025
Address:
Suzhou, China
Editors:
Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, Violet Peng
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
16915–16941
Language:
URL:
https://preview.aclanthology.org/name-variant-enfa-fane/2025.emnlp-main.856/
DOI:
10.18653/v1/2025.emnlp-main.856
Bibkey:
Cite (ACL):
Junho Yoo and Youhyun Shin. 2025. Fair or Framed? Political Bias in News Articles Generated by LLMs. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 16915–16941, Suzhou, China. Association for Computational Linguistics.
Cite (Informal):
Fair or Framed? Political Bias in News Articles Generated by LLMs (Yoo & Shin, EMNLP 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/name-variant-enfa-fane/2025.emnlp-main.856.pdf
Checklist:
 2025.emnlp-main.856.checklist.pdf