Abstract
Growing literature has shown that NLP systems may encode social biases; however, the *political* bias of summarization models remains relatively unknown. In this work, we use an entity replacement method to investigate the portrayal of politicians in automatically generated summaries of news articles. We develop an entity-based computational framework to assess the sensitivities of several extractive and abstractive summarizers to the politicians Donald Trump and Joe Biden. We find consistent differences in these summaries upon entity replacement, such as reduced emphasis of Trump’s presence in the context of the same article and a more individualistic representation of Trump with respect to the collective US government (i.e., administration). These summary dissimilarities are most prominent when the entity is heavily featured in the source article. Our characterization provides a foundation for future studies of bias in summarization and for normative discussions on the ideal qualities of automatic summaries.- Anthology ID:
- 2023.findings-emnlp.696
- Volume:
- Findings of the Association for Computational Linguistics: EMNLP 2023
- Month:
- December
- Year:
- 2023
- Address:
- Singapore
- Editors:
- Houda Bouamor, Juan Pino, Kalika Bali
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 10374–10386
- Language:
- URL:
- https://aclanthology.org/2023.findings-emnlp.696
- DOI:
- 10.18653/v1/2023.findings-emnlp.696
- Cite (ACL):
- Karen Zhou and Chenhao Tan. 2023. Entity-Based Evaluation of Political Bias in Automatic Summarization. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 10374–10386, Singapore. Association for Computational Linguistics.
- Cite (Informal):
- Entity-Based Evaluation of Political Bias in Automatic Summarization (Zhou & Tan, Findings 2023)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2023.findings-emnlp.696.pdf