The Pluralistic Moral Gap: Understanding Moral Judgment and Value Differences between Humans and Large Language Models

Giuseppe Russo, Debora Nozza, Paul Röttger, Dirk Hovy


Abstract
People increasingly rely on Large Language Models (LLMs) for moral advice, which may influence humans’ decisions. Yet, little is known about how closely LLMs align with human moral judgments. To address this, we introduce the Moral Dilemma Dataset, a benchmark of 1,618 real-world moral dilemmas paired with a distribution of human moral judgments consisting of a binary evaluation and a free-text rationale. We treat this problem as a pluralistic distributional alignment task, comparing the distributions of LLM and human judgments across dilemmas. We find that models reproduce human judgments only under high consensus; alignment deteriorates sharply when human disagreement increases. In parallel, using a 60-value taxonomy built from 3,783 value expressions extracted from rationales, we show that LLMs rely on a narrower set of moral values than humans. These findings reveal a pluralistic moral gap–a mismatch in both the distribution and diversity of values expressed. To close this gap, we introduce Dynamic Moral Profiling (DMP), a Dirichlet-based sampling method that conditions model outputs on human-derived value profiles. DMP improves alignment by 64.3% and enhances value diversity, offering a step toward more pluralistic and human-aligned moral guidance.
Anthology ID:
2026.eacl-long.305
Volume:
Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
March
Year:
2026
Address:
Rabat, Morocco
Editors:
Vera Demberg, Kentaro Inui, Lluís Marquez
Venue:
EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
6481–6497
Language:
URL:
https://preview.aclanthology.org/ingest-eacl/2026.eacl-long.305/
DOI:
Bibkey:
Cite (ACL):
Giuseppe Russo, Debora Nozza, Paul Röttger, and Dirk Hovy. 2026. The Pluralistic Moral Gap: Understanding Moral Judgment and Value Differences between Humans and Large Language Models. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6481–6497, Rabat, Morocco. Association for Computational Linguistics.
Cite (Informal):
The Pluralistic Moral Gap: Understanding Moral Judgment and Value Differences between Humans and Large Language Models (Russo et al., EACL 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-eacl/2026.eacl-long.305.pdf